[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:55:55 dist-test-slave-f9xb krb5kdc[7326](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:55:55 dist-test-slave-f9xb krb5kdc[7326](info): set up 2 sockets
Sep 26 14:55:55 dist-test-slave-f9xb krb5kdc[7326](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260926 14:55:55.951898 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.921s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:55:56 dist-test-slave-f9xb krb5kdc[7326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:55:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:55:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:55:56.054198 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:37941
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:37941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:55:56.128808 7339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:55:56.128978 7339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:55:56.128997 7339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:55:56.130069 7339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:55:56.130105 7339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:55:56.130117 7339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:55:56.130129 7339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:55:56.130141 7339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:55:56.131531 7339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:37941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:37941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:55:56.131731 7339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:55:56.131937 7339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:55:56.136375 7347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:55:56.136490 7339 server_base.cc:1034] running on GCE node
W20260926 14:55:56.136384 7345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:55:56.136411 7344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:55:56.136761 7339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:55:56.137037 7339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:55:56.138190 7339 hybrid_clock.cc:648] HybridClock initialized: now 1790434556138170 us; error 34 us; skew 500 ppm
Sep 26 14:55:56 dist-test-slave-f9xb krb5kdc[7326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434556, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:55:56.140547 7339 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:55:56.140962 7339 webserver.cc:460] Webserver started at http://127.7.37.254:38641/ using document root <none> and password file <none>
I20260926 14:55:56.141153 7339 fs_manager.cc:359] Metadata directory not provided
I20260926 14:55:56.141199 7339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:55:56.141314 7339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:55:56.142216 7339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "aa3c9ed0d99f480198aed964ab97845b"
format_stamp: "Formatted at 2026-09-26 14:55:56 on dist-test-slave-f9xb"
server_key: "6decb55308fbbcf6385aea2ef33b005c"
server_key_iv: "9761b307ade087348c19b663f60bd297"
server_key_version: "encryptionkey@0"
I20260926 14:55:56.142517 7339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "aa3c9ed0d99f480198aed964ab97845b"
format_stamp: "Formatted at 2026-09-26 14:55:56 on dist-test-slave-f9xb"
server_key: "6decb55308fbbcf6385aea2ef33b005c"
server_key_iv: "9761b307ade087348c19b663f60bd297"
server_key_version: "encryptionkey@0"
I20260926 14:55:56.143702 7339 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:55:56.144353 7354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:56.144495 7339 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:55:56.144570 7339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "aa3c9ed0d99f480198aed964ab97845b"
format_stamp: "Formatted at 2026-09-26 14:55:56 on dist-test-slave-f9xb"
server_key: "6decb55308fbbcf6385aea2ef33b005c"
server_key_iv: "9761b307ade087348c19b663f60bd297"
server_key_version: "encryptionkey@0"
I20260926 14:55:56.144627 7339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:55:56.158366 7339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:55:56.160300 7339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:55:56.160414 7339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:55:56.163851 7339 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:37941
I20260926 14:55:56.163890 7406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:37941 every 8 connection(s)
I20260926 14:55:56.164160 7339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:55:56.164870 7407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:55:56.166829 7407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b: Bootstrap starting.
I20260926 14:55:56.167296 7407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b: Neither blocks nor log segments found. Creating new log.
I20260926 14:55:56.167578 7407 log.cc:824] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b: Log is configured to *not* fsync() on all Append() calls
I20260926 14:55:56.167829 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 7339
I20260926 14:55:56.167912 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:55:56.168212 7407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b: No bootstrap required, opened a new log
I20260926 14:55:56.168483 7319 external_mini_cluster.cc:1428] Setting key 47c69f7922d196dc1270c004d9112a76
I20260926 14:55:56.169823 7407 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3c9ed0d99f480198aed964ab97845b" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37941 } }
I20260926 14:55:56.169946 7407 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:55:56.169967 7407 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa3c9ed0d99f480198aed964ab97845b, State: Initialized, Role: FOLLOWER
I20260926 14:55:56.170068 7407 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3c9ed0d99f480198aed964ab97845b" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37941 } }
I20260926 14:55:56.170128 7407 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:55:56.170149 7407 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:55:56.170199 7407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:55:56.170907 7407 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3c9ed0d99f480198aed964ab97845b" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37941 } }
I20260926 14:55:56.170986 7407 leader_election.cc:302] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa3c9ed0d99f480198aed964ab97845b; no voters:
I20260926 14:55:56.171130 7407 leader_election.cc:288] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:55:56.171169 7412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:55:56.171314 7412 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b [term 1 LEADER]: Becoming Leader. State: Replica: aa3c9ed0d99f480198aed964ab97845b, State: Running, Role: LEADER
I20260926 14:55:56.171372 7407 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:55:56.171414 7412 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3c9ed0d99f480198aed964ab97845b" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37941 } }
I20260926 14:55:56.171723 7413 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa3c9ed0d99f480198aed964ab97845b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3c9ed0d99f480198aed964ab97845b" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37941 } } }
I20260926 14:55:56.171813 7413 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b [sys.catalog]: This master's current role is: LEADER
I20260926 14:55:56.171957 7414 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa3c9ed0d99f480198aed964ab97845b. Latest consensus state: current_term: 1 leader_uuid: "aa3c9ed0d99f480198aed964ab97845b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3c9ed0d99f480198aed964ab97845b" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37941 } } }
I20260926 14:55:56.172015 7414 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b [sys.catalog]: This master's current role is: LEADER
I20260926 14:55:56.172132 7418 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:55:56.172618 7418 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:55:56.174106 7418 catalog_manager.cc:1347] Generated new cluster ID: 73729685a4664e23a3242f35c1962e54
I20260926 14:55:56.174155 7418 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 14:55:56 dist-test-slave-f9xb krb5kdc[7326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:55:56.181773 7411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:56.169176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41594 (local address 127.7.37.254:37941)
0926 14:55:56.169442 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:56.169450 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:56.169503 (+ 53us) server_negotiation.cc:407] Connection header received
0926 14:55:56.171195 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:56.171203 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:56.171271 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:56.173337 (+ 2066us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:55:56.173417 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:56.174485 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:56.175420 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:56.175881 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:56.176663 (+ 782us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:56.178805 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:55:56.178808 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:55:56.178808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:55:56.178817 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:55:56.180223 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:56.180568 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:56.180571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:56.180571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:56.180657 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:56.180875 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:56.180875 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:56.180876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:56.181085 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:55:56.181224 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:56.181457 (+ 233us) server_negotiation.cc:299] Negotiation successful
0926 14:55:56.181572 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":69,"threads_started":1}
I20260926 14:55:56.188143 7418 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:55:56.188572 7418 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:55:56.201061 7418 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa3c9ed0d99f480198aed964ab97845b: Generated new TSK 0
I20260926 14:55:56.201188 7418 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
I20260926 14:55:56.230188 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:37941
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:55:56.307734 7433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:55:56.307926 7433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:55:56.307947 7433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:55:56.309052 7433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:55:56.309091 7433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:55:56.309121 7433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:55:56.310514 7433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:37941
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:55:56.310724 7433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:55:56.310909 7433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:55:56.315253 7441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:55:56.315260 7439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:55:56.315325 7438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:55:56.315359 7433 server_base.cc:1034] running on GCE node
I20260926 14:55:56.315595 7433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:55:56.315825 7433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:55:56.316964 7433 hybrid_clock.cc:648] HybridClock initialized: now 1790434556316951 us; error 31 us; skew 500 ppm
Sep 26 14:55:56 dist-test-slave-f9xb krb5kdc[7326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434556, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:55:56.319113 7433 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:55:56.319492 7433 webserver.cc:460] Webserver started at http://127.7.37.193:45539/ using document root <none> and password file <none>
I20260926 14:55:56.319679 7433 fs_manager.cc:359] Metadata directory not provided
I20260926 14:55:56.319725 7433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:55:56.319854 7433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:55:56.320757 7433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "a518eb8d1e1f4b3aa7d322d688bfdbae"
format_stamp: "Formatted at 2026-09-26 14:55:56 on dist-test-slave-f9xb"
server_key: "51f796c3aafaccd0a7234a0f76ab1052"
server_key_iv: "99efa5536a48c381f7c62d0e62be8b72"
server_key_version: "encryptionkey@0"
I20260926 14:55:56.321023 7433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "a518eb8d1e1f4b3aa7d322d688bfdbae"
format_stamp: "Formatted at 2026-09-26 14:55:56 on dist-test-slave-f9xb"
server_key: "51f796c3aafaccd0a7234a0f76ab1052"
server_key_iv: "99efa5536a48c381f7c62d0e62be8b72"
server_key_version: "encryptionkey@0"
I20260926 14:55:56.322089 7433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:55:56.322649 7448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:56.322793 7433 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:55:56.322867 7433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "a518eb8d1e1f4b3aa7d322d688bfdbae"
format_stamp: "Formatted at 2026-09-26 14:55:56 on dist-test-slave-f9xb"
server_key: "51f796c3aafaccd0a7234a0f76ab1052"
server_key_iv: "99efa5536a48c381f7c62d0e62be8b72"
server_key_version: "encryptionkey@0"
I20260926 14:55:56.322923 7433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:55:56.337749 7433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:55:56.339628 7433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:55:56.339768 7433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:55:56.339944 7433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:55:56.340189 7433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:55:56.340225 7433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:56.340250 7433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:55:56.340273 7433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:56.345463 7433 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:43407
I20260926 14:55:56.345494 7561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:43407 every 8 connection(s)
I20260926 14:55:56.345743 7433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
Sep 26 14:55:56 dist-test-slave-f9xb krb5kdc[7326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434556, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:55:56.353506 7411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:56.346284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:46285 (local address 127.7.37.254:37941)
0926 14:55:56.346349 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:56.346351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:56.346502 (+ 151us) server_negotiation.cc:407] Connection header received
0926 14:55:56.346898 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:56.346900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:56.346914 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:56.346972 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:55:56.347834 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:56.348393 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:56.349256 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:56.349438 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:56.349486 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:56.351206 (+ 1720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:55:56.351207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:55:56.351207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:55:56.351214 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:55:56.352567 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:56.352878 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:56.352879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:56.352879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:56.352922 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:56.353079 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:56.353079 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:56.353079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:56.353173 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:55:56.353215 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:56.353396 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 14:55:56.353447 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 14:55:56.353628 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:56.346253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:37941 (local address 127.7.37.193:46285)
0926 14:55:56.346374 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 14:55:56.346425 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 14:55:56.346816 (+ 391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:55:56.347019 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:55:56.347022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:55:56.347565 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:55:56.347734 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:56.347738 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:56.348451 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:56.348451 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:56.349187 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:56.349198 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:56.349492 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:56.349492 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:56.349578 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:56.349925 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:55:56.349932 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:55:56.351127 (+ 1195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:55:56.352638 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:55:56.352644 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:55:56.352645 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:55:56.352828 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:55:56.352963 (+ 135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:55:56.352963 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:55:56.352964 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:55:56.353039 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:55:56.353214 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:55:56.353216 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:55:56.353309 (+ 93us) client_negotiation.cc:769] Sending connection context
0926 14:55:56.353377 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 14:55:56.353476 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":67,"threads_started":1}
I20260926 14:55:56.353876 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 7433
I20260926 14:55:56.353960 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:55:56.354058 7562 heartbeater.cc:347] Connected to a master server at 127.7.37.254:37941
I20260926 14:55:56.354096 7319 external_mini_cluster.cc:1428] Setting key 7bddbce980d0e6fa8d0960255c813a78
I20260926 14:55:56.354138 7562 heartbeater.cc:464] Registering TS with master...
I20260926 14:55:56.354295 7562 heartbeater.cc:511] Master 127.7.37.254:37941 requested a full tablet report, sending...
I20260926 14:55:56.354794 7371 ts_manager.cc:194] Registered new tserver with Master: a518eb8d1e1f4b3aa7d322d688bfdbae (127.7.37.193:43407)
I20260926 14:55:56.355568 7371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:46285
WARNING: no policy specified for kudu/127.7.37.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
I20260926 14:55:56.374539 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:37941
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:55:56.449770 7567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:55:56.449947 7567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:55:56.449968 7567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:55:56.451040 7567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:55:56.451076 7567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:55:56.451104 7567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:55:56.452574 7567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:37941
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:55:56.452786 7567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:55:56.452976 7567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:55:56.457306 7573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:55:56.457402 7567 server_base.cc:1034] running on GCE node
W20260926 14:55:56.457317 7575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:55:56.457773 7572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:55:56.457959 7567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:55:56.458173 7567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:55:56.459308 7567 hybrid_clock.cc:648] HybridClock initialized: now 1790434556459296 us; error 27 us; skew 500 ppm
Sep 26 14:55:56 dist-test-slave-f9xb krb5kdc[7326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434556, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:55:56.461580 7567 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:55:56.461979 7567 webserver.cc:460] Webserver started at http://127.7.37.194:40943/ using document root <none> and password file <none>
I20260926 14:55:56.462167 7567 fs_manager.cc:359] Metadata directory not provided
I20260926 14:55:56.462224 7567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:55:56.462342 7567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:55:56.463261 7567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "cdf4a1a7626d42a9a1b20b961d8d5030"
format_stamp: "Formatted at 2026-09-26 14:55:56 on dist-test-slave-f9xb"
server_key: "6ca38633a02d30940defb4b572a566ae"
server_key_iv: "8ac548e637f073081fcffe9648432c33"
server_key_version: "encryptionkey@0"
I20260926 14:55:56.463567 7567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "cdf4a1a7626d42a9a1b20b961d8d5030"
format_stamp: "Formatted at 2026-09-26 14:55:56 on dist-test-slave-f9xb"
server_key: "6ca38633a02d30940defb4b572a566ae"
server_key_iv: "8ac548e637f073081fcffe9648432c33"
server_key_version: "encryptionkey@0"
I20260926 14:55:56.464887 7567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260926 14:55:56.465560 7582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:56.465694 7567 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:55:56.465767 7567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "cdf4a1a7626d42a9a1b20b961d8d5030"
format_stamp: "Formatted at 2026-09-26 14:55:56 on dist-test-slave-f9xb"
server_key: "6ca38633a02d30940defb4b572a566ae"
server_key_iv: "8ac548e637f073081fcffe9648432c33"
server_key_version: "encryptionkey@0"
I20260926 14:55:56.465830 7567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:55:56.485227 7567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:55:56.487057 7567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:55:56.487185 7567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:55:56.487366 7567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:55:56.487603 7567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:55:56.487633 7567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:56.487658 7567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:55:56.487681 7567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:56.493101 7567 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:40487
I20260926 14:55:56.493150 7695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:40487 every 8 connection(s)
I20260926 14:55:56.493395 7567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:55:56.498332 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 7567
I20260926 14:55:56.498421 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
Sep 26 14:55:56 dist-test-slave-f9xb krb5kdc[7326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434556, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:55:56.498565 7319 external_mini_cluster.cc:1428] Setting key 4689ac198a071abe27c59e9f588f4c84
I20260926 14:55:56.501281 7411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:56.493960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:48903 (local address 127.7.37.254:37941)
0926 14:55:56.494003 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:56.494004 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:56.494159 (+ 155us) server_negotiation.cc:407] Connection header received
0926 14:55:56.494549 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:56.494550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:56.494558 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:56.494606 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:55:56.495366 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:56.495890 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:56.496797 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:56.496982 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:56.497033 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:56.498850 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:55:56.498852 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:55:56.498852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:55:56.498859 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:55:56.500122 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:56.500462 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:56.500463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:56.500464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:56.500514 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:56.500722 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:56.500723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:56.500723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:56.500824 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:55:56.500877 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:56.501107 (+ 230us) server_negotiation.cc:299] Negotiation successful
0926 14:55:56.501194 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:55:56.501399 7698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:56.493935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:37941 (local address 127.7.37.194:48903)
0926 14:55:56.494064 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 14:55:56.494099 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 14:55:56.494482 (+ 383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:55:56.494652 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:55:56.494655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:55:56.495136 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:55:56.495305 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:56.495309 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:56.495948 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:56.495949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:56.496719 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:56.496722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:56.497039 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:56.497039 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:56.497122 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:56.497515 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:55:56.497522 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:55:56.498775 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:55:56.500197 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:55:56.500203 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:55:56.500205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:55:56.500400 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:55:56.500564 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:55:56.500565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:55:56.500565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:55:56.500662 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:55:56.500875 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:55:56.500880 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:55:56.500998 (+ 118us) client_negotiation.cc:769] Sending connection context
0926 14:55:56.501080 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 14:55:56.501212 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":71,"threads_started":1}
I20260926 14:55:56.501767 7696 heartbeater.cc:347] Connected to a master server at 127.7.37.254:37941
I20260926 14:55:56.501844 7696 heartbeater.cc:464] Registering TS with master...
I20260926 14:55:56.501994 7696 heartbeater.cc:511] Master 127.7.37.254:37941 requested a full tablet report, sending...
I20260926 14:55:56.502315 7371 ts_manager.cc:194] Registered new tserver with Master: cdf4a1a7626d42a9a1b20b961d8d5030 (127.7.37.194:40487)
I20260926 14:55:56.502794 7371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:48903
WARNING: no policy specified for kudu/127.7.37.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
I20260926 14:55:56.519008 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:37941
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:55:56.594771 7701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:55:56.594938 7701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:55:56.594959 7701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:55:56.596011 7701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:55:56.596043 7701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:55:56.596067 7701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:55:56.597455 7701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:37941
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:55:56.597616 7701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:55:56.597764 7701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:55:56.602035 7706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:55:56.602141 7701 server_base.cc:1034] running on GCE node
W20260926 14:55:56.602128 7709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:55:56.602049 7707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:55:56.602409 7701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:55:56.602603 7701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:55:56.603721 7701 hybrid_clock.cc:648] HybridClock initialized: now 1790434556603713 us; error 28 us; skew 500 ppm
Sep 26 14:55:56 dist-test-slave-f9xb krb5kdc[7326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434556, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:55:56.605830 7701 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:55:56.606191 7701 webserver.cc:460] Webserver started at http://127.7.37.195:37119/ using document root <none> and password file <none>
I20260926 14:55:56.606360 7701 fs_manager.cc:359] Metadata directory not provided
I20260926 14:55:56.606396 7701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:55:56.606478 7701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:55:56.607332 7701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "206ae6a281844f8a86c753fc652c4a9b"
format_stamp: "Formatted at 2026-09-26 14:55:56 on dist-test-slave-f9xb"
server_key: "d54fbae11b27c38ef13f6dbef934a535"
server_key_iv: "3e44d260c2821d07d81d45515fe4059a"
server_key_version: "encryptionkey@0"
I20260926 14:55:56.607596 7701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "206ae6a281844f8a86c753fc652c4a9b"
format_stamp: "Formatted at 2026-09-26 14:55:56 on dist-test-slave-f9xb"
server_key: "d54fbae11b27c38ef13f6dbef934a535"
server_key_iv: "3e44d260c2821d07d81d45515fe4059a"
server_key_version: "encryptionkey@0"
I20260926 14:55:56.608727 7701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:55:56.609308 7716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:56.609462 7701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:55:56.609540 7701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "206ae6a281844f8a86c753fc652c4a9b"
format_stamp: "Formatted at 2026-09-26 14:55:56 on dist-test-slave-f9xb"
server_key: "d54fbae11b27c38ef13f6dbef934a535"
server_key_iv: "3e44d260c2821d07d81d45515fe4059a"
server_key_version: "encryptionkey@0"
I20260926 14:55:56.609596 7701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:55:56.616335 7701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:55:56.618314 7701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:55:56.618419 7701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:55:56.618587 7701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:55:56.618799 7701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:55:56.618829 7701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:56.618856 7701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:55:56.618875 7701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:56.624886 7701 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:41307
I20260926 14:55:56.624940 7829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:41307 every 8 connection(s)
I20260926 14:55:56.625185 7701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
Sep 26 14:55:56 dist-test-slave-f9xb krb5kdc[7326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434556, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:55:56.632479 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 7701
I20260926 14:55:56.632592 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:55:56.632750 7319 external_mini_cluster.cc:1428] Setting key ff6590cb310de9a4db154794d31e8f1f
I20260926 14:55:56.633215 7411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:56.625748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:34507 (local address 127.7.37.254:37941)
0926 14:55:56.625797 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:56.625799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:56.626036 (+ 237us) server_negotiation.cc:407] Connection header received
0926 14:55:56.626448 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:56.626449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:56.626461 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:56.626534 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:55:56.627344 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:56.627889 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:56.628811 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:56.629011 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:56.629064 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:56.630844 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:55:56.630845 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:55:56.630846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:55:56.630853 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:55:56.631973 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:56.632307 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:56.632308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:56.632309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:56.632373 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:56.632570 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:56.632572 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:56.632572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:56.632702 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:55:56.632761 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:56.633048 (+ 287us) server_negotiation.cc:299] Negotiation successful
0926 14:55:56.633133 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 14:55:56.633389 7832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:56.625672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:37941 (local address 127.7.37.195:34507)
0926 14:55:56.625922 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0926 14:55:56.625960 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 14:55:56.626364 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:55:56.626593 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:55:56.626596 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:55:56.627087 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:55:56.627258 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:56.627262 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:56.627970 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:56.627971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:56.628732 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:56.628734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:56.629073 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:56.629074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:56.629193 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:56.629596 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:55:56.629603 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:55:56.630767 (+ 1164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:55:56.632045 (+ 1278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:55:56.632051 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:55:56.632053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:55:56.632247 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:55:56.632424 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:55:56.632425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:55:56.632425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:55:56.632518 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:55:56.632759 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:55:56.632765 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:55:56.632901 (+ 136us) client_negotiation.cc:769] Sending connection context
0926 14:55:56.632992 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 14:55:56.633138 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":172,"threads_started":1}
I20260926 14:55:56.633760 7830 heartbeater.cc:347] Connected to a master server at 127.7.37.254:37941
I20260926 14:55:56.633841 7830 heartbeater.cc:464] Registering TS with master...
I20260926 14:55:56.633997 7830 heartbeater.cc:511] Master 127.7.37.254:37941 requested a full tablet report, sending...
I20260926 14:55:56.634300 7371 ts_manager.cc:194] Registered new tserver with Master: 206ae6a281844f8a86c753fc652c4a9b (127.7.37.195:41307)
I20260926 14:55:56.634610 7371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:34507
I20260926 14:55:56.644651 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 14:55:56 dist-test-slave-f9xb krb5kdc[7326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434556, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 14:55:56 dist-test-slave-f9xb krb5kdc[7326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434556, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.37.193@KRBTEST.COM
I20260926 14:55:56.664463 7837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:56.658034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59942 (local address 127.7.37.193:43407)
0926 14:55:56.658190 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:56.658193 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:56.658202 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:55:56.658223 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:56.658224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:56.658236 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:56.658294 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:55:56.658740 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:56.659412 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:56.660114 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:56.660405 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:56.660454 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:56.662317 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:55:56.662323 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:55:56.662324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:55:56.662329 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:55:56.663655 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:56.663935 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:56.663937 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:56.663937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:56.663987 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:56.664107 (+ 120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:56.664107 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:56.664108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:56.664210 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:55:56.664309 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:56.664341 (+ 32us) server_negotiation.cc:299] Negotiation successful
0926 14:55:56.664397 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
W20260926 14:55:56.664745 7476 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59942
Sep 26 14:55:56 dist-test-slave-f9xb krb5kdc[7326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 14:55:56 dist-test-slave-f9xb krb5kdc[7326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.193@KRBTEST.COM
I20260926 14:55:56.682881 7837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:56.677173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59958 (local address 127.7.37.193:43407)
0926 14:55:56.677224 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:56.677226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:56.677280 (+ 54us) server_negotiation.cc:407] Connection header received
0926 14:55:56.677340 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:56.677341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:56.677347 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:56.677389 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:55:56.677861 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:56.678297 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:56.678938 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:56.679108 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:56.679146 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:56.681080 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:55:56.681081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:55:56.681081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:55:56.681086 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:55:56.682098 (+ 1012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:56.682368 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:56.682368 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:56.682369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:56.682408 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:56.682541 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:56.682541 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:56.682542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:56.682614 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:55:56.682698 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:56.682761 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 14:55:56.682814 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 14:55:56.683303 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 7433
I20260926 14:55:56.688194 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 7567
I20260926 14:55:56.691486 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 7701
I20260926 14:55:56.694856 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 7339
2026-09-26T14:55:56Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1687 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:55:56 dist-test-slave-f9xb krb5kdc[7844](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:55:56 dist-test-slave-f9xb krb5kdc[7844](info): set up 2 sockets
Sep 26 14:55:56 dist-test-slave-f9xb krb5kdc[7844](info): commencing operation
krb5kdc: starting...
W20260926 14:55:57.630842 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.917s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:55:57 dist-test-slave-f9xb krb5kdc[7844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:55:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:55:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:55:57.730131 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:34213
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:34213
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:55:57.804535 7857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:55:57.804704 7857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:55:57.804723 7857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:55:57.805819 7857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:55:57.805857 7857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:55:57.805871 7857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:55:57.805892 7857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:55:57.805917 7857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:55:57.807358 7857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:34213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:34213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:55:57.807576 7857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:55:57.807865 7857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:55:57.812047 7863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:55:57.812078 7865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:55:57.812071 7862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:55:57.812264 7857 server_base.cc:1034] running on GCE node
I20260926 14:55:57.812493 7857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:55:57.812724 7857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:55:57.813859 7857 hybrid_clock.cc:648] HybridClock initialized: now 1790434557813842 us; error 35 us; skew 500 ppm
Sep 26 14:55:57 dist-test-slave-f9xb krb5kdc[7844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434557, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:55:57.820079 7857 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:55:57.820407 7857 webserver.cc:460] Webserver started at http://127.7.37.254:43119/ using document root <none> and password file <none>
I20260926 14:55:57.820569 7857 fs_manager.cc:359] Metadata directory not provided
I20260926 14:55:57.820611 7857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:55:57.820690 7857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:55:57.821576 7857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "4022e5114557461a9eb208e5730ce979"
format_stamp: "Formatted at 2026-09-26 14:55:57 on dist-test-slave-f9xb"
server_key: "9cbc9cbf998de9541fd57bdfdd1b38fa"
server_key_iv: "73ef3aaf96ec01d5e0be744e684ae577"
server_key_version: "encryptionkey@0"
I20260926 14:55:57.821835 7857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "4022e5114557461a9eb208e5730ce979"
format_stamp: "Formatted at 2026-09-26 14:55:57 on dist-test-slave-f9xb"
server_key: "9cbc9cbf998de9541fd57bdfdd1b38fa"
server_key_iv: "73ef3aaf96ec01d5e0be744e684ae577"
server_key_version: "encryptionkey@0"
I20260926 14:55:57.822887 7857 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:55:57.823482 7872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:57.823616 7857 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:57.823683 7857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "4022e5114557461a9eb208e5730ce979"
format_stamp: "Formatted at 2026-09-26 14:55:57 on dist-test-slave-f9xb"
server_key: "9cbc9cbf998de9541fd57bdfdd1b38fa"
server_key_iv: "73ef3aaf96ec01d5e0be744e684ae577"
server_key_version: "encryptionkey@0"
I20260926 14:55:57.823762 7857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:55:57.865165 7857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:55:57.867130 7857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:55:57.867233 7857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:55:57.870682 7857 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:34213
I20260926 14:55:57.870726 7924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:34213 every 8 connection(s)
I20260926 14:55:57.871004 7857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:55:57.871785 7925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:55:57.873704 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 7857
I20260926 14:55:57.873776 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:55:57.873799 7925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979: Bootstrap starting.
I20260926 14:55:57.873924 7319 external_mini_cluster.cc:1428] Setting key b696b695b3a7c37e35ff51f5f73112d0
I20260926 14:55:57.874400 7925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979: Neither blocks nor log segments found. Creating new log.
I20260926 14:55:57.874656 7925 log.cc:824] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979: Log is configured to *not* fsync() on all Append() calls
I20260926 14:55:57.875294 7925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979: No bootstrap required, opened a new log
I20260926 14:55:57.877281 7925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4022e5114557461a9eb208e5730ce979" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34213 } }
I20260926 14:55:57.877435 7925 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:55:57.877465 7925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4022e5114557461a9eb208e5730ce979, State: Initialized, Role: FOLLOWER
I20260926 14:55:57.877576 7925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4022e5114557461a9eb208e5730ce979" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34213 } }
I20260926 14:55:57.877651 7925 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:55:57.877691 7925 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:55:57.877735 7925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:55:57.878234 7925 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4022e5114557461a9eb208e5730ce979" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34213 } }
I20260926 14:55:57.878341 7925 leader_election.cc:302] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4022e5114557461a9eb208e5730ce979; no voters:
I20260926 14:55:57.878490 7925 leader_election.cc:288] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:55:57.878556 7930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:55:57.878712 7925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:55:57.878743 7930 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979 [term 1 LEADER]: Becoming Leader. State: Replica: 4022e5114557461a9eb208e5730ce979, State: Running, Role: LEADER
I20260926 14:55:57.878899 7930 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4022e5114557461a9eb208e5730ce979" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34213 } }
I20260926 14:55:57.879189 7930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4022e5114557461a9eb208e5730ce979. Latest consensus state: current_term: 1 leader_uuid: "4022e5114557461a9eb208e5730ce979" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4022e5114557461a9eb208e5730ce979" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34213 } } }
I20260926 14:55:57.879267 7930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979 [sys.catalog]: This master's current role is: LEADER
I20260926 14:55:57.879200 7932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4022e5114557461a9eb208e5730ce979" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4022e5114557461a9eb208e5730ce979" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34213 } } }
I20260926 14:55:57.879350 7932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979 [sys.catalog]: This master's current role is: LEADER
I20260926 14:55:57.879618 7939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:55:57.880077 7939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 14:55:57 dist-test-slave-f9xb krb5kdc[7844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:55:57.881651 7939 catalog_manager.cc:1347] Generated new cluster ID: 5e6483628cf84fd3a6cbd87229f11b1b
I20260926 14:55:57.881699 7939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:55:57.886098 7928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:57.874343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57584 (local address 127.7.37.254:34213)
0926 14:55:57.874517 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:57.874526 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:57.874551 (+ 25us) server_negotiation.cc:407] Connection header received
0926 14:55:57.874970 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:57.874976 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:57.875019 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:57.875129 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:55:57.875812 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:57.876649 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:57.877634 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:57.878089 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:57.878165 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:57.880790 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:55:57.880793 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:55:57.880793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:55:57.880804 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:55:57.884625 (+ 3821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:57.884969 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:57.884973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:57.884974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:57.885094 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:57.885277 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:57.885278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:57.885278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:57.885558 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:55:57.885705 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:57.885780 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 14:55:57.885904 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":58,"threads_started":1}
I20260926 14:55:57.890486 7939 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:55:57.891129 7939 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:55:57.899039 7939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4022e5114557461a9eb208e5730ce979: Generated new TSK 0
I20260926 14:55:57.899178 7939 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
I20260926 14:55:57.925773 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:34213
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:55:58.000514 7951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:55:58.000697 7951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:55:58.000731 7951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:55:58.001794 7951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:55:58.001827 7951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:55:58.001850 7951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:55:58.003208 7951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:34213
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:55:58.003368 7951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:55:58.003535 7951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:55:58.007881 7956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:55:58.007910 7959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:55:58.007910 7957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:55:58.008107 7951 server_base.cc:1034] running on GCE node
I20260926 14:55:58.008322 7951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:55:58.008533 7951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:55:58.009680 7951 hybrid_clock.cc:648] HybridClock initialized: now 1790434558009665 us; error 24 us; skew 500 ppm
Sep 26 14:55:58 dist-test-slave-f9xb krb5kdc[7844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434558, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:55:58.011963 7951 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:55:58.012338 7951 webserver.cc:460] Webserver started at http://127.7.37.193:36135/ using document root <none> and password file <none>
I20260926 14:55:58.012517 7951 fs_manager.cc:359] Metadata directory not provided
I20260926 14:55:58.012562 7951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:55:58.012676 7951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:55:58.013592 7951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "eb815baa402743ef993cb770841c0317"
format_stamp: "Formatted at 2026-09-26 14:55:58 on dist-test-slave-f9xb"
server_key: "156e31c086fa3e96b49c91b999df2d92"
server_key_iv: "1d115995b942cd162de55dfdb4648490"
server_key_version: "encryptionkey@0"
I20260926 14:55:58.013909 7951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "eb815baa402743ef993cb770841c0317"
format_stamp: "Formatted at 2026-09-26 14:55:58 on dist-test-slave-f9xb"
server_key: "156e31c086fa3e96b49c91b999df2d92"
server_key_iv: "1d115995b942cd162de55dfdb4648490"
server_key_version: "encryptionkey@0"
I20260926 14:55:58.015100 7951 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 14:55:58.015766 7966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:58.015914 7951 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:55:58.015990 7951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "eb815baa402743ef993cb770841c0317"
format_stamp: "Formatted at 2026-09-26 14:55:58 on dist-test-slave-f9xb"
server_key: "156e31c086fa3e96b49c91b999df2d92"
server_key_iv: "1d115995b942cd162de55dfdb4648490"
server_key_version: "encryptionkey@0"
I20260926 14:55:58.016052 7951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:55:58.027592 7951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:55:58.029156 7951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:55:58.029281 7951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:55:58.029455 7951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:55:58.029700 7951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:55:58.029731 7951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:58.029755 7951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:55:58.029772 7951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:58.034792 7951 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:45069
I20260926 14:55:58.034842 8079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:45069 every 8 connection(s)
I20260926 14:55:58.035074 7951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:55:58.039363 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 7951
I20260926 14:55:58.039443 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:55:58.039572 7319 external_mini_cluster.cc:1428] Setting key 3f441beaacd014bc9eb6bb93b3f507b8
Sep 26 14:55:58 dist-test-slave-f9xb krb5kdc[7844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434558, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:55:58.043020 7928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.035638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:52759 (local address 127.7.37.254:34213)
0926 14:55:58.035685 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:58.035687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:58.035869 (+ 182us) server_negotiation.cc:407] Connection header received
0926 14:55:58.036687 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:58.036689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:58.036697 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:58.036745 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:55:58.037224 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.037709 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.038592 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.038772 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.038814 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.040602 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:55:58.040603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:55:58.040603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:55:58.040610 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:55:58.041842 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:58.042200 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:58.042201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:58.042201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:58.042255 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:58.042465 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:58.042465 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:58.042466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:58.042591 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:55:58.042649 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:58.042841 (+ 192us) server_negotiation.cc:299] Negotiation successful
0926 14:55:58.042925 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 14:55:58.043048 8082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.035611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34213 (local address 127.7.37.193:52759)
0926 14:55:58.035770 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 14:55:58.035805 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 14:55:58.036602 (+ 797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:55:58.036792 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:55:58.036795 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:55:58.037003 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:55:58.037167 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:58.037170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.037776 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.037777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:58.038533 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:58.038535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.038822 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.038822 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:58.038908 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.039250 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:55:58.039256 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:55:58.040521 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:55:58.041914 (+ 1393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:55:58.041921 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:55:58.041923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:55:58.042136 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:55:58.042322 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:55:58.042322 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:55:58.042323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:55:58.042413 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:55:58.042642 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:55:58.042645 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:55:58.042740 (+ 95us) client_negotiation.cc:769] Sending connection context
0926 14:55:58.042811 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 14:55:58.042915 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":64,"threads_started":1}
I20260926 14:55:58.043421 8080 heartbeater.cc:347] Connected to a master server at 127.7.37.254:34213
I20260926 14:55:58.043493 8080 heartbeater.cc:464] Registering TS with master...
I20260926 14:55:58.043618 8080 heartbeater.cc:511] Master 127.7.37.254:34213 requested a full tablet report, sending...
I20260926 14:55:58.044034 7889 ts_manager.cc:194] Registered new tserver with Master: eb815baa402743ef993cb770841c0317 (127.7.37.193:45069)
I20260926 14:55:58.044601 7889 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:52759
WARNING: no policy specified for kudu/127.7.37.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
I20260926 14:55:58.060161 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:34213
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:55:58.136880 8085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:55:58.137075 8085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:55:58.137105 8085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:55:58.138206 8085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:55:58.138243 8085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:55:58.138278 8085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:55:58.139679 8085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:34213
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:55:58.139904 8085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:55:58.140101 8085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:55:58.144413 8091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:55:58.144459 8090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:55:58.144428 8093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:55:58.144688 8085 server_base.cc:1034] running on GCE node
I20260926 14:55:58.144881 8085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:55:58.145082 8085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:55:58.146229 8085 hybrid_clock.cc:648] HybridClock initialized: now 1790434558146216 us; error 27 us; skew 500 ppm
Sep 26 14:55:58 dist-test-slave-f9xb krb5kdc[7844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434558, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:55:58.148433 8085 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:55:58.148824 8085 webserver.cc:460] Webserver started at http://127.7.37.194:40057/ using document root <none> and password file <none>
I20260926 14:55:58.149011 8085 fs_manager.cc:359] Metadata directory not provided
I20260926 14:55:58.149055 8085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:55:58.149168 8085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:55:58.150079 8085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "131ed1e25c674b6096c46597d97cf691"
format_stamp: "Formatted at 2026-09-26 14:55:58 on dist-test-slave-f9xb"
server_key: "1ab46fde62126d326cda15d3312ed430"
server_key_iv: "ac87587271ac8d1edecf86acffad7c2e"
server_key_version: "encryptionkey@0"
I20260926 14:55:58.150385 8085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "131ed1e25c674b6096c46597d97cf691"
format_stamp: "Formatted at 2026-09-26 14:55:58 on dist-test-slave-f9xb"
server_key: "1ab46fde62126d326cda15d3312ed430"
server_key_iv: "ac87587271ac8d1edecf86acffad7c2e"
server_key_version: "encryptionkey@0"
I20260926 14:55:58.151546 8085 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 14:55:58.152216 8100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:58.152357 8085 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:58.152429 8085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "131ed1e25c674b6096c46597d97cf691"
format_stamp: "Formatted at 2026-09-26 14:55:58 on dist-test-slave-f9xb"
server_key: "1ab46fde62126d326cda15d3312ed430"
server_key_iv: "ac87587271ac8d1edecf86acffad7c2e"
server_key_version: "encryptionkey@0"
I20260926 14:55:58.152489 8085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:55:58.164350 8085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:55:58.165905 8085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:55:58.166024 8085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:55:58.166188 8085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:55:58.166417 8085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:55:58.166446 8085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:58.166471 8085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:55:58.166494 8085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:58.171530 8085 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:33783
I20260926 14:55:58.171582 8213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:33783 every 8 connection(s)
I20260926 14:55:58.171844 8085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:55:58.173775 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 8085
I20260926 14:55:58.173851 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:55:58.173981 7319 external_mini_cluster.cc:1428] Setting key 309e45f44838471846f03ff91b04fe1a
Sep 26 14:55:58 dist-test-slave-f9xb krb5kdc[7844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434558, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:55:58.180857 7928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.172443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:49295 (local address 127.7.37.254:34213)
0926 14:55:58.172485 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:58.172487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:58.172664 (+ 177us) server_negotiation.cc:407] Connection header received
0926 14:55:58.173493 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:58.173495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:58.173507 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:58.173571 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:55:58.174203 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.174775 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.175855 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.176054 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.176110 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.178186 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:55:58.178187 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:55:58.178188 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:55:58.178195 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:55:58.179598 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:58.179970 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:58.179972 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:58.179972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:58.180038 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:58.180233 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:58.180234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:58.180234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:58.180362 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:55:58.180427 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:58.180691 (+ 264us) server_negotiation.cc:299] Negotiation successful
0926 14:55:58.180769 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 14:55:58.180891 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.172401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34213 (local address 127.7.37.194:49295)
0926 14:55:58.172562 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 14:55:58.172591 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 14:55:58.173394 (+ 803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:55:58.173655 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:55:58.173659 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:55:58.173894 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:55:58.174121 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:58.174127 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.174854 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.174855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:58.175780 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:58.175784 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.176122 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.176123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:58.176252 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.176684 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:55:58.176693 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:55:58.178106 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:55:58.179662 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:55:58.179667 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:55:58.179669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:55:58.179908 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:55:58.180097 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:55:58.180098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:55:58.180098 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:55:58.180181 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:55:58.180450 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:55:58.180455 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:55:58.180574 (+ 119us) client_negotiation.cc:769] Sending connection context
0926 14:55:58.180651 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 14:55:58.180755 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
I20260926 14:55:58.181509 8214 heartbeater.cc:347] Connected to a master server at 127.7.37.254:34213
I20260926 14:55:58.181584 8214 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.37.195@KRBTEST.COM; defaulting to no policy
I20260926 14:55:58.181738 8214 heartbeater.cc:511] Master 127.7.37.254:34213 requested a full tablet report, sending...
I20260926 14:55:58.182055 7889 ts_manager.cc:194] Registered new tserver with Master: 131ed1e25c674b6096c46597d97cf691 (127.7.37.194:33783)
I20260926 14:55:58.182371 7889 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:49295
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
I20260926 14:55:58.196281 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:34213
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:55:58.271940 8219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:55:58.272118 8219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:55:58.272140 8219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:55:58.273262 8219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:55:58.273299 8219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:55:58.273326 8219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:55:58.274739 8219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:34213
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:55:58.274943 8219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:55:58.275132 8219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:55:58.279561 8224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:55:58.279580 8225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:55:58.279778 8227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:55:58.279960 8219 server_base.cc:1034] running on GCE node
I20260926 14:55:58.280184 8219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:55:58.280386 8219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:55:58.281507 8219 hybrid_clock.cc:648] HybridClock initialized: now 1790434558281495 us; error 29 us; skew 500 ppm
Sep 26 14:55:58 dist-test-slave-f9xb krb5kdc[7844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434558, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:55:58.283625 8219 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:55:58.284056 8219 webserver.cc:460] Webserver started at http://127.7.37.195:40039/ using document root <none> and password file <none>
I20260926 14:55:58.284248 8219 fs_manager.cc:359] Metadata directory not provided
I20260926 14:55:58.284293 8219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:55:58.284410 8219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:55:58.285313 8219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "7164e9c826b44b48a1f89e87db621e73"
format_stamp: "Formatted at 2026-09-26 14:55:58 on dist-test-slave-f9xb"
server_key: "d6505f5df6347217d8561f556a3abc0c"
server_key_iv: "38373fd06d937af40a2542c3bc172f58"
server_key_version: "encryptionkey@0"
I20260926 14:55:58.285620 8219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "7164e9c826b44b48a1f89e87db621e73"
format_stamp: "Formatted at 2026-09-26 14:55:58 on dist-test-slave-f9xb"
server_key: "d6505f5df6347217d8561f556a3abc0c"
server_key_iv: "38373fd06d937af40a2542c3bc172f58"
server_key_version: "encryptionkey@0"
I20260926 14:55:58.286739 8219 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 14:55:58.287328 8234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:58.287468 8219 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:55:58.287540 8219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "7164e9c826b44b48a1f89e87db621e73"
format_stamp: "Formatted at 2026-09-26 14:55:58 on dist-test-slave-f9xb"
server_key: "d6505f5df6347217d8561f556a3abc0c"
server_key_iv: "38373fd06d937af40a2542c3bc172f58"
server_key_version: "encryptionkey@0"
I20260926 14:55:58.287604 8219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:55:58.314164 8219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:55:58.315718 8219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:55:58.315863 8219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:55:58.316043 8219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:55:58.316294 8219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:55:58.316324 8219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:58.316350 8219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:55:58.316373 8219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:58.321658 8219 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:33349
I20260926 14:55:58.321723 8347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:33349 every 8 connection(s)
I20260926 14:55:58.321961 8219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
Sep 26 14:55:58 dist-test-slave-f9xb krb5kdc[7844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434558, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:55:58.329883 7928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.322529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:41451 (local address 127.7.37.254:34213)
0926 14:55:58.322566 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:58.322568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:58.322748 (+ 180us) server_negotiation.cc:407] Connection header received
0926 14:55:58.323573 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:58.323574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:58.323583 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:58.323634 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:55:58.324158 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.324639 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.325579 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.325807 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.325858 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.327774 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:55:58.327776 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:55:58.327776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:55:58.327784 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:55:58.328799 (+ 1015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:58.329110 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:58.329111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:58.329111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:58.329151 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:58.329355 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:58.329356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:58.329356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:58.329476 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:55:58.329575 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:58.329759 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 14:55:58.329822 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 14:55:58.330039 8350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.322507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34213 (local address 127.7.37.195:41451)
0926 14:55:58.322648 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 14:55:58.322685 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 14:55:58.323480 (+ 795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:55:58.323692 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:55:58.323696 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:55:58.323929 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:55:58.324096 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:58.324100 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.324708 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.324709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:58.325468 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:58.325470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.325866 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.325867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:58.325968 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.326371 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:55:58.326380 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:55:58.327666 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:55:58.328861 (+ 1195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:55:58.328866 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:55:58.328867 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:55:58.329056 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:55:58.329193 (+ 137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:55:58.329194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:55:58.329194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:55:58.329279 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:55:58.329566 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:55:58.329568 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:55:58.329663 (+ 95us) client_negotiation.cc:769] Sending connection context
0926 14:55:58.329734 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 14:55:58.329833 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
I20260926 14:55:58.330019 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 8219
I20260926 14:55:58.330123 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:55:58.330273 7319 external_mini_cluster.cc:1428] Setting key fc7a7577dc1e583df27c357f40109626
I20260926 14:55:58.330452 8348 heartbeater.cc:347] Connected to a master server at 127.7.37.254:34213
I20260926 14:55:58.330574 8348 heartbeater.cc:464] Registering TS with master...
I20260926 14:55:58.330747 8348 heartbeater.cc:511] Master 127.7.37.254:34213 requested a full tablet report, sending...
I20260926 14:55:58.331108 7889 ts_manager.cc:194] Registered new tserver with Master: 7164e9c826b44b48a1f89e87db621e73 (127.7.37.195:33349)
I20260926 14:55:58.331470 7889 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:41451
I20260926 14:55:58.341734 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 14:55:58.348308 7928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.343060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57598 (local address 127.7.37.254:34213)
0926 14:55:58.343124 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:58.343126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:58.343198 (+ 72us) server_negotiation.cc:407] Connection header received
0926 14:55:58.343245 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:58.343246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:58.343254 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:58.343303 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:55:58.343820 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.344264 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.344919 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.345110 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.345160 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.346163 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:55:58.346164 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:55:58.346164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:55:58.346171 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:55:58.347194 (+ 1023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:58.347534 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:58.347535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:58.347535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:58.347592 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:58.347791 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:58.347792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:58.347792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:58.347923 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:55:58.348013 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:58.348144 (+ 131us) server_negotiation.cc:299] Negotiation successful
0926 14:55:58.348227 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 14:55:58.350162 7889 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57598:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:55:58.351637 7889 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:55:58.357429 8357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.353447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:33783 (local address 127.0.0.1:51222)
0926 14:55:58.353713 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0926 14:55:58.353751 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 14:55:58.353816 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:55:58.354038 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:55:58.354039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:55:58.354041 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:55:58.354097 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:58.354099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.355194 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.355195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:58.356449 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:58.356452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.357146 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.357147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:58.357223 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.357235 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 14:55:58.357303 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 14:55:58.357350 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":53,"threads_started":1}
I20260926 14:55:58.357427 8358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.353555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51222 (local address 127.7.37.194:33783)
0926 14:55:58.353894 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:58.353899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:58.353909 (+ 10us) server_negotiation.cc:407] Connection header received
0926 14:55:58.353944 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:58.353946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:58.353958 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:58.354032 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:55:58.354183 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.355102 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.356552 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.357068 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.357138 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.357195 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:58.357317 (+ 122us) server_negotiation.cc:299] Negotiation successful
0926 14:55:58.357354 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":68,"threads_started":1}
I20260926 14:55:58.357717 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.353457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:45069 (local address 127.0.0.1:42344)
0926 14:55:58.353736 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0926 14:55:58.353765 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 14:55:58.353817 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:55:58.354012 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:55:58.354015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:55:58.354017 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:55:58.354072 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:58.354074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.355478 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.355479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:58.356557 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:58.356559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.357479 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.357481 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:58.357556 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.357558 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:55:58.357597 (+ 39us) client_negotiation.cc:240] Negotiation successful
0926 14:55:58.357634 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":61,"threads_started":1}
I20260926 14:55:58.357762 8361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.353712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60680 (local address 127.7.37.195:33349)
0926 14:55:58.353853 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:58.353856 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:58.354198 (+ 342us) server_negotiation.cc:407] Connection header received
0926 14:55:58.354272 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:58.354274 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:58.354288 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:58.354356 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:55:58.354471 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.355471 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.356661 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.357333 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.357397 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.357446 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:58.357670 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 14:55:58.357703 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
I20260926 14:55:58.357880 8360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.353636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:33349 (local address 127.0.0.1:60680)
0926 14:55:58.354140 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0926 14:55:58.354150 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:55:58.354201 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:55:58.354363 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:55:58.354364 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:55:58.354366 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:55:58.354400 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:58.354401 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.355673 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.355674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:58.356586 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:58.356588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.357481 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.357482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:58.357545 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.357548 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:55:58.357596 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 14:55:58.357633 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":4,"thread_start_us":59,"threads_started":1}
I20260926 14:55:58.357929 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.353579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42344 (local address 127.7.37.193:45069)
0926 14:55:58.353749 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:58.353753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:58.353886 (+ 133us) server_negotiation.cc:407] Connection header received
0926 14:55:58.353925 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:58.353926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:58.353938 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:58.353998 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:55:58.354141 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.355379 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.356622 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.357334 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.357398 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.357446 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:58.357817 (+ 371us) server_negotiation.cc:299] Negotiation successful
0926 14:55:58.357860 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":67,"threads_started":1}
I20260926 14:55:58.358304 8148 tablet_service.cc:1467] Processing CreateTablet for tablet d6fd273f907543afb6460937a51f590f (DEFAULT_TABLE table=test-table [id=9558873e5229413692a06b3295cf8820]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:55:58.358475 8014 tablet_service.cc:1467] Processing CreateTablet for tablet d6fd273f907543afb6460937a51f590f (DEFAULT_TABLE table=test-table [id=9558873e5229413692a06b3295cf8820]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:55:58.358637 8148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fd273f907543afb6460937a51f590f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:55:58.358788 8014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fd273f907543afb6460937a51f590f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:55:58.360146 8282 tablet_service.cc:1467] Processing CreateTablet for tablet d6fd273f907543afb6460937a51f590f (DEFAULT_TABLE table=test-table [id=9558873e5229413692a06b3295cf8820]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:55:58.360416 8282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fd273f907543afb6460937a51f590f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:55:58.361191 8362 tablet_bootstrap.cc:492] T d6fd273f907543afb6460937a51f590f P eb815baa402743ef993cb770841c0317: Bootstrap starting.
I20260926 14:55:58.361689 8363 tablet_bootstrap.cc:492] T d6fd273f907543afb6460937a51f590f P 131ed1e25c674b6096c46597d97cf691: Bootstrap starting.
I20260926 14:55:58.362290 8362 tablet_bootstrap.cc:654] T d6fd273f907543afb6460937a51f590f P eb815baa402743ef993cb770841c0317: Neither blocks nor log segments found. Creating new log.
I20260926 14:55:58.362323 8363 tablet_bootstrap.cc:654] T d6fd273f907543afb6460937a51f590f P 131ed1e25c674b6096c46597d97cf691: Neither blocks nor log segments found. Creating new log.
I20260926 14:55:58.362712 8362 log.cc:824] T d6fd273f907543afb6460937a51f590f P eb815baa402743ef993cb770841c0317: Log is configured to *not* fsync() on all Append() calls
I20260926 14:55:58.362869 8363 log.cc:824] T d6fd273f907543afb6460937a51f590f P 131ed1e25c674b6096c46597d97cf691: Log is configured to *not* fsync() on all Append() calls
I20260926 14:55:58.363307 8362 tablet_bootstrap.cc:492] T d6fd273f907543afb6460937a51f590f P eb815baa402743ef993cb770841c0317: No bootstrap required, opened a new log
I20260926 14:55:58.363385 8366 tablet_bootstrap.cc:492] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73: Bootstrap starting.
I20260926 14:55:58.363425 8362 ts_tablet_manager.cc:1397] T d6fd273f907543afb6460937a51f590f P eb815baa402743ef993cb770841c0317: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:55:58.363584 8363 tablet_bootstrap.cc:492] T d6fd273f907543afb6460937a51f590f P 131ed1e25c674b6096c46597d97cf691: No bootstrap required, opened a new log
I20260926 14:55:58.363641 8363 ts_tablet_manager.cc:1397] T d6fd273f907543afb6460937a51f590f P 131ed1e25c674b6096c46597d97cf691: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:55:58.363955 8366 tablet_bootstrap.cc:654] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73: Neither blocks nor log segments found. Creating new log.
I20260926 14:55:58.364445 8366 log.cc:824] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73: Log is configured to *not* fsync() on all Append() calls
I20260926 14:55:58.365130 8366 tablet_bootstrap.cc:492] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73: No bootstrap required, opened a new log
I20260926 14:55:58.365204 8366 ts_tablet_manager.cc:1397] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:55:58.365306 8362 raft_consensus.cc:348] T d6fd273f907543afb6460937a51f590f P eb815baa402743ef993cb770841c0317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb815baa402743ef993cb770841c0317" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45069 } } peers { permanent_uuid: "131ed1e25c674b6096c46597d97cf691" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 33783 } } peers { permanent_uuid: "7164e9c826b44b48a1f89e87db621e73" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33349 } }
I20260926 14:55:58.365429 8362 raft_consensus.cc:374] T d6fd273f907543afb6460937a51f590f P eb815baa402743ef993cb770841c0317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:55:58.365290 8363 raft_consensus.cc:348] T d6fd273f907543afb6460937a51f590f P 131ed1e25c674b6096c46597d97cf691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb815baa402743ef993cb770841c0317" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45069 } } peers { permanent_uuid: "131ed1e25c674b6096c46597d97cf691" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 33783 } } peers { permanent_uuid: "7164e9c826b44b48a1f89e87db621e73" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33349 } }
I20260926 14:55:58.365466 8362 raft_consensus.cc:729] T d6fd273f907543afb6460937a51f590f P eb815baa402743ef993cb770841c0317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb815baa402743ef993cb770841c0317, State: Initialized, Role: FOLLOWER
I20260926 14:55:58.365500 8363 raft_consensus.cc:374] T d6fd273f907543afb6460937a51f590f P 131ed1e25c674b6096c46597d97cf691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:55:58.365550 8363 raft_consensus.cc:729] T d6fd273f907543afb6460937a51f590f P 131ed1e25c674b6096c46597d97cf691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 131ed1e25c674b6096c46597d97cf691, State: Initialized, Role: FOLLOWER
I20260926 14:55:58.365592 8362 consensus_queue.cc:260] T d6fd273f907543afb6460937a51f590f P eb815baa402743ef993cb770841c0317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb815baa402743ef993cb770841c0317" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45069 } } peers { permanent_uuid: "131ed1e25c674b6096c46597d97cf691" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 33783 } } peers { permanent_uuid: "7164e9c826b44b48a1f89e87db621e73" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33349 } }
I20260926 14:55:58.365661 8363 consensus_queue.cc:260] T d6fd273f907543afb6460937a51f590f P 131ed1e25c674b6096c46597d97cf691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb815baa402743ef993cb770841c0317" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45069 } } peers { permanent_uuid: "131ed1e25c674b6096c46597d97cf691" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 33783 } } peers { permanent_uuid: "7164e9c826b44b48a1f89e87db621e73" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33349 } }
I20260926 14:55:58.365826 8362 ts_tablet_manager.cc:1428] T d6fd273f907543afb6460937a51f590f P eb815baa402743ef993cb770841c0317: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:55:58.365916 8363 ts_tablet_manager.cc:1428] T d6fd273f907543afb6460937a51f590f P 131ed1e25c674b6096c46597d97cf691: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 14:55:58.366060 8080 heartbeater.cc:503] Master 127.7.37.254:34213 was elected leader, sending a full tablet report...
I20260926 14:55:58.366413 8214 heartbeater.cc:503] Master 127.7.37.254:34213 was elected leader, sending a full tablet report...
I20260926 14:55:58.367406 8366 raft_consensus.cc:348] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb815baa402743ef993cb770841c0317" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45069 } } peers { permanent_uuid: "131ed1e25c674b6096c46597d97cf691" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 33783 } } peers { permanent_uuid: "7164e9c826b44b48a1f89e87db621e73" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33349 } }
I20260926 14:55:58.367530 8366 raft_consensus.cc:374] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:55:58.367553 8366 raft_consensus.cc:729] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7164e9c826b44b48a1f89e87db621e73, State: Initialized, Role: FOLLOWER
I20260926 14:55:58.367631 8366 consensus_queue.cc:260] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb815baa402743ef993cb770841c0317" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45069 } } peers { permanent_uuid: "131ed1e25c674b6096c46597d97cf691" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 33783 } } peers { permanent_uuid: "7164e9c826b44b48a1f89e87db621e73" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33349 } }
I20260926 14:55:58.367888 8366 ts_tablet_manager.cc:1428] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260926 14:55:58.367923 8348 heartbeater.cc:503] Master 127.7.37.254:34213 was elected leader, sending a full tablet report...
W20260926 14:55:58.422295 8215 tablet.cc:2369] T d6fd273f907543afb6460937a51f590f P 131ed1e25c674b6096c46597d97cf691: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:55:58.505230 8370 raft_consensus.cc:482] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:55:58.505322 8370 raft_consensus.cc:504] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb815baa402743ef993cb770841c0317" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45069 } } peers { permanent_uuid: "131ed1e25c674b6096c46597d97cf691" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 33783 } } peers { permanent_uuid: "7164e9c826b44b48a1f89e87db621e73" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33349 } }
I20260926 14:55:58.505663 8370 leader_election.cc:288] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb815baa402743ef993cb770841c0317 (127.7.37.193:45069), 131ed1e25c674b6096c46597d97cf691 (127.7.37.194:33783)
I20260926 14:55:58.508123 8350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.505812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:33783 (local address 127.7.37.195:52425)
0926 14:55:58.505848 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0926 14:55:58.505859 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:55:58.505897 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:55:58.506037 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:55:58.506038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:55:58.506039 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:55:58.506103 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:58.506105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.506672 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.506673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:58.507383 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:58.507385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.507907 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.507908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:58.507976 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.507979 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:55:58.508022 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 14:55:58.508057 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 14:55:58.508126 8358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.505904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:52425 (local address 127.7.37.194:33783)
0926 14:55:58.505946 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:58.505948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:58.505955 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:55:58.505977 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:58.505978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:58.505986 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:58.506029 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:55:58.506160 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.506605 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.507469 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.507840 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.507899 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.507906 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:58.508039 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 14:55:58.508072 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 14:55:58.508462 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.505866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:59531 (local address 127.7.37.193:45069)
0926 14:55:58.505910 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:58.505912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:58.506054 (+ 142us) server_negotiation.cc:407] Connection header received
0926 14:55:58.506111 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:58.506112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:58.506125 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:58.506172 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:55:58.506305 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.506758 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.507683 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.508104 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.508173 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.508181 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:58.508386 (+ 205us) server_negotiation.cc:299] Negotiation successful
0926 14:55:58.508417 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 14:55:58.508495 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.505814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:45069 (local address 127.7.37.195:59531)
0926 14:55:58.505980 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 14:55:58.505990 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:55:58.506048 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:55:58.506180 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:55:58.506180 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:55:58.506182 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:55:58.506246 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:58.506247 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.506835 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.506836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:58.507597 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:58.507599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.508245 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.508246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:58.508320 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.508324 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:55:58.508382 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 14:55:58.508420 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":11,"thread_start_us":88,"threads_started":1}
I20260926 14:55:58.508466 8168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fd273f907543afb6460937a51f590f" candidate_uuid: "7164e9c826b44b48a1f89e87db621e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "131ed1e25c674b6096c46597d97cf691" is_pre_election: true
I20260926 14:55:58.508613 8168 raft_consensus.cc:2454] T d6fd273f907543afb6460937a51f590f P 131ed1e25c674b6096c46597d97cf691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7164e9c826b44b48a1f89e87db621e73 in term 0.
I20260926 14:55:58.508822 8235 leader_election.cc:302] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 131ed1e25c674b6096c46597d97cf691, 7164e9c826b44b48a1f89e87db621e73; no voters:
I20260926 14:55:58.508837 8034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fd273f907543afb6460937a51f590f" candidate_uuid: "7164e9c826b44b48a1f89e87db621e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb815baa402743ef993cb770841c0317" is_pre_election: true
I20260926 14:55:58.508960 8034 raft_consensus.cc:2454] T d6fd273f907543afb6460937a51f590f P eb815baa402743ef993cb770841c0317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7164e9c826b44b48a1f89e87db621e73 in term 0.
I20260926 14:55:58.508956 8370 raft_consensus.cc:2781] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:55:58.509014 8370 raft_consensus.cc:482] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:55:58.509035 8370 raft_consensus.cc:3037] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:55:58.509652 8370 raft_consensus.cc:504] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb815baa402743ef993cb770841c0317" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45069 } } peers { permanent_uuid: "131ed1e25c674b6096c46597d97cf691" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 33783 } } peers { permanent_uuid: "7164e9c826b44b48a1f89e87db621e73" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33349 } }
I20260926 14:55:58.509754 8370 leader_election.cc:288] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73 [CANDIDATE]: Term 1 election: Requested vote from peers eb815baa402743ef993cb770841c0317 (127.7.37.193:45069), 131ed1e25c674b6096c46597d97cf691 (127.7.37.194:33783)
I20260926 14:55:58.509896 8034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fd273f907543afb6460937a51f590f" candidate_uuid: "7164e9c826b44b48a1f89e87db621e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb815baa402743ef993cb770841c0317"
I20260926 14:55:58.509925 8168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fd273f907543afb6460937a51f590f" candidate_uuid: "7164e9c826b44b48a1f89e87db621e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "131ed1e25c674b6096c46597d97cf691"
I20260926 14:55:58.509963 8034 raft_consensus.cc:3037] T d6fd273f907543afb6460937a51f590f P eb815baa402743ef993cb770841c0317 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:55:58.509984 8168 raft_consensus.cc:3037] T d6fd273f907543afb6460937a51f590f P 131ed1e25c674b6096c46597d97cf691 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:55:58.510571 8168 raft_consensus.cc:2454] T d6fd273f907543afb6460937a51f590f P 131ed1e25c674b6096c46597d97cf691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7164e9c826b44b48a1f89e87db621e73 in term 1.
I20260926 14:55:58.510684 8034 raft_consensus.cc:2454] T d6fd273f907543afb6460937a51f590f P eb815baa402743ef993cb770841c0317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7164e9c826b44b48a1f89e87db621e73 in term 1.
I20260926 14:55:58.510717 8235 leader_election.cc:302] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 131ed1e25c674b6096c46597d97cf691, 7164e9c826b44b48a1f89e87db621e73; no voters:
I20260926 14:55:58.510794 8370 raft_consensus.cc:2781] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:55:58.510912 8370 raft_consensus.cc:686] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73 [term 1 LEADER]: Becoming Leader. State: Replica: 7164e9c826b44b48a1f89e87db621e73, State: Running, Role: LEADER
I20260926 14:55:58.510994 8370 consensus_queue.cc:237] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb815baa402743ef993cb770841c0317" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45069 } } peers { permanent_uuid: "131ed1e25c674b6096c46597d97cf691" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 33783 } } peers { permanent_uuid: "7164e9c826b44b48a1f89e87db621e73" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33349 } }
I20260926 14:55:58.511780 7889 catalog_manager.cc:5495] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7164e9c826b44b48a1f89e87db621e73 (127.7.37.195). New cstate: current_term: 1 leader_uuid: "7164e9c826b44b48a1f89e87db621e73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb815baa402743ef993cb770841c0317" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "131ed1e25c674b6096c46597d97cf691" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 33783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7164e9c826b44b48a1f89e87db621e73" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33349 } health_report { overall_health: HEALTHY } } }
W20260926 14:55:58.535666 8081 tablet.cc:2369] T d6fd273f907543afb6460937a51f590f P eb815baa402743ef993cb770841c0317: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 14:55:58 dist-test-slave-f9xb krb5kdc[7844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.193@KRBTEST.COM
I20260926 14:55:58.538295 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.532087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42354 (local address 127.7.37.193:45069)
0926 14:55:58.532127 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:58.532129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:58.532217 (+ 88us) server_negotiation.cc:407] Connection header received
0926 14:55:58.532273 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:58.532273 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:58.532282 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:58.532325 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:55:58.532782 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.533206 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.533850 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.534040 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.534091 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.536163 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:55:58.536168 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:55:58.536168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:55:58.536174 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:55:58.537413 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:58.537715 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:58.537717 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:58.537717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:58.537763 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:58.537897 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:58.537897 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:58.537898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:58.538017 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:55:58.538124 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:58.538175 (+ 51us) server_negotiation.cc:299] Negotiation successful
0926 14:55:58.538241 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Sep 26 14:55:58 dist-test-slave-f9xb krb5kdc[7844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434558, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 14:55:58 dist-test-slave-f9xb krb5kdc[7844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434558, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.37.193@KRBTEST.COM
I20260926 14:55:58.558669 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.552678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42370 (local address 127.7.37.193:45069)
0926 14:55:58.552719 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:58.552720 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:58.552810 (+ 90us) server_negotiation.cc:407] Connection header received
0926 14:55:58.552866 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:58.552867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:58.552875 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:58.552917 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:55:58.553380 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.553797 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.554479 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.554666 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.554718 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.556832 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:55:58.556833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:55:58.556833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:55:58.556839 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:55:58.557944 (+ 1105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:58.558192 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:58.558192 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:58.558192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:58.558230 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:58.558340 (+ 110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:58.558341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:58.558341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:58.558405 (+ 64us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:55:58.558462 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:58.558567 (+ 105us) server_negotiation.cc:299] Negotiation successful
0926 14:55:58.558615 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Sep 26 14:55:58 dist-test-slave-f9xb krb5kdc[7844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260926 14:55:58.572415 8349 tablet.cc:2369] T d6fd273f907543afb6460937a51f590f P 7164e9c826b44b48a1f89e87db621e73: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 14:55:58 dist-test-slave-f9xb krb5kdc[7844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.193@KRBTEST.COM
I20260926 14:55:58.577481 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:58.571656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42378 (local address 127.7.37.193:45069)
0926 14:55:58.571713 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:58.571715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:58.571774 (+ 59us) server_negotiation.cc:407] Connection header received
0926 14:55:58.571836 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:58.571836 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:58.571845 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:58.571885 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:55:58.572326 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.572789 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.573440 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:58.573624 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:58.573671 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:58.575468 (+ 1797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:55:58.575469 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:55:58.575469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:55:58.575475 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:55:58.576608 (+ 1133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:58.576876 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:58.576876 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:58.576877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:58.576921 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:58.577068 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:58.577068 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:58.577069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:58.577165 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:55:58.577240 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:58.577345 (+ 105us) server_negotiation.cc:299] Negotiation successful
0926 14:55:58.577405 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260926 14:55:58.577828 7994 tablet_service.cc:2908] Rejecting scan request for tablet d6fd273f907543afb6460937a51f590f: Uninitialized: clean time has not yet been initialized
I20260926 14:55:58.578439 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 7951
I20260926 14:55:58.582500 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 8085
I20260926 14:55:58.586189 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 8219
I20260926 14:55:58.589787 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 7857
2026-09-26T14:55:58Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1894 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:55:58 dist-test-slave-f9xb krb5kdc[8386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:55:58 dist-test-slave-f9xb krb5kdc[8386](info): set up 2 sockets
Sep 26 14:55:58 dist-test-slave-f9xb krb5kdc[8386](info): commencing operation
krb5kdc: starting...
W20260926 14:55:59.530848 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.923s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:55:59 dist-test-slave-f9xb krb5kdc[8386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:55:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:55:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:55:59.634059 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:46637
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:37587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:46637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:55:59.707947 8399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:55:59.708115 8399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:55:59.708135 8399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:55:59.709167 8399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:55:59.709200 8399 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 14:55:59.709213 8399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:55:59.709224 8399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:55:59.709234 8399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:55:59.709244 8399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:55:59.710578 8399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:37587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:46637
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:46637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:55:59.710747 8399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:55:59.710893 8399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:55:59.715080 8407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:55:59.715183 8399 server_base.cc:1034] running on GCE node
W20260926 14:55:59.715101 8404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:55:59.715101 8405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:55:59.715459 8399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:55:59.715705 8399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:55:59.716867 8399 hybrid_clock.cc:648] HybridClock initialized: now 1790434559716850 us; error 33 us; skew 500 ppm
Sep 26 14:55:59 dist-test-slave-f9xb krb5kdc[8386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434559, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:55:59.718926 8399 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:55:59.719272 8399 webserver.cc:460] Webserver started at http://127.7.37.254:39291/ using document root <none> and password file <none>
I20260926 14:55:59.719429 8399 fs_manager.cc:359] Metadata directory not provided
I20260926 14:55:59.719463 8399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:55:59.719543 8399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:55:59.720448 8399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "04a7136da95d44f29413545a9cad927f"
format_stamp: "Formatted at 2026-09-26 14:55:59 on dist-test-slave-f9xb"
server_key: "86562a7522a849c4519a5f9d85a8c058"
server_key_iv: "520687327556fc664daccbe76c2a610d"
server_key_version: "encryptionkey@0"
I20260926 14:55:59.720712 8399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "04a7136da95d44f29413545a9cad927f"
format_stamp: "Formatted at 2026-09-26 14:55:59 on dist-test-slave-f9xb"
server_key: "86562a7522a849c4519a5f9d85a8c058"
server_key_iv: "520687327556fc664daccbe76c2a610d"
server_key_version: "encryptionkey@0"
I20260926 14:55:59.721742 8399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 14:55:59.722334 8414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:59.722463 8399 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:55:59.722536 8399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "04a7136da95d44f29413545a9cad927f"
format_stamp: "Formatted at 2026-09-26 14:55:59 on dist-test-slave-f9xb"
server_key: "86562a7522a849c4519a5f9d85a8c058"
server_key_iv: "520687327556fc664daccbe76c2a610d"
server_key_version: "encryptionkey@0"
I20260926 14:55:59.722596 8399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:55:59.732189 8399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:55:59.733708 8399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:55:59.733822 8399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:55:59.737792 8399 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:46637
I20260926 14:55:59.737839 8476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:46637 every 8 connection(s)
I20260926 14:55:59.738083 8399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:55:59.738621 8477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:55:59.740583 8477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f: Bootstrap starting.
I20260926 14:55:59.741091 8477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f: Neither blocks nor log segments found. Creating new log.
I20260926 14:55:59.741333 8477 log.cc:824] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f: Log is configured to *not* fsync() on all Append() calls
I20260926 14:55:59.741906 8477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f: No bootstrap required, opened a new log
I20260926 14:55:59.743839 8477 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04a7136da95d44f29413545a9cad927f" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 46637 } }
I20260926 14:55:59.743959 8477 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:55:59.743979 8477 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a7136da95d44f29413545a9cad927f, State: Initialized, Role: FOLLOWER
I20260926 14:55:59.744099 8477 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04a7136da95d44f29413545a9cad927f" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 46637 } }
I20260926 14:55:59.744154 8477 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:55:59.744174 8477 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:55:59.744215 8477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:55:59.744678 8477 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04a7136da95d44f29413545a9cad927f" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 46637 } }
I20260926 14:55:59.744757 8477 leader_election.cc:302] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04a7136da95d44f29413545a9cad927f; no voters:
I20260926 14:55:59.744896 8477 leader_election.cc:288] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:55:59.744936 8480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:55:59.745071 8480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f [term 1 LEADER]: Becoming Leader. State: Replica: 04a7136da95d44f29413545a9cad927f, State: Running, Role: LEADER
I20260926 14:55:59.745105 8477 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:55:59.745167 8480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04a7136da95d44f29413545a9cad927f" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 46637 } }
I20260926 14:55:59.745493 8481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04a7136da95d44f29413545a9cad927f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04a7136da95d44f29413545a9cad927f" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 46637 } } }
I20260926 14:55:59.745499 8482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04a7136da95d44f29413545a9cad927f. Latest consensus state: current_term: 1 leader_uuid: "04a7136da95d44f29413545a9cad927f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04a7136da95d44f29413545a9cad927f" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 46637 } } }
I20260926 14:55:59.745659 8481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f [sys.catalog]: This master's current role is: LEADER
I20260926 14:55:59.745672 8482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f [sys.catalog]: This master's current role is: LEADER
I20260926 14:55:59.745864 8489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:55:59.746342 8489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:55:59.747295 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 8399
I20260926 14:55:59.747370 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:55:59.747496 7319 external_mini_cluster.cc:1428] Setting key ac7c005f088263ee7bb075b7af82ea72
I20260926 14:55:59.752544 8489 catalog_manager.cc:1347] Generated new cluster ID: bd65c03f125b4adea67726f562980c94
I20260926 14:55:59.752592 8489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 14:55:59 dist-test-slave-f9xb krb5kdc[8386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:55:59.757063 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:59.747870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42284 (local address 127.7.37.254:46637)
0926 14:55:59.748000 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:59.748008 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:59.748040 (+ 32us) server_negotiation.cc:407] Connection header received
0926 14:55:59.748126 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:59.748130 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:59.748171 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:59.748270 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:55:59.748980 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:59.749854 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:59.750870 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:59.751264 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:59.751336 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:59.753811 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:55:59.753815 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:55:59.753815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:55:59.753826 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:55:59.755565 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:59.755928 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:59.755931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:59.755931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:59.756041 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:59.756229 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:59.756230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:59.756230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:59.756507 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:55:59.756655 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:59.756726 (+ 71us) server_negotiation.cc:299] Negotiation successful
0926 14:55:59.756850 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":62,"threads_started":1}
I20260926 14:55:59.769331 8489 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:55:59.769996 8489 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:55:59.785511 8489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f: Generated new TSK 0
I20260926 14:55:59.785655 8489 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
I20260926 14:55:59.814071 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:46637
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:37587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:55:59.888365 8503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:55:59.888532 8503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:55:59.888553 8503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:55:59.889640 8503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:55:59.889679 8503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:55:59.889698 8503 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 14:55:59.889714 8503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:55:59.891042 8503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:37587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.37.254:46637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:55:59.891251 8503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:55:59.891441 8503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:55:59.895869 8508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:55:59.895792 8511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:55:59.895805 8509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:55:59.895891 8503 server_base.cc:1034] running on GCE node
I20260926 14:55:59.896150 8503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:55:59.896332 8503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:55:59.897460 8503 hybrid_clock.cc:648] HybridClock initialized: now 1790434559897448 us; error 26 us; skew 500 ppm
Sep 26 14:55:59 dist-test-slave-f9xb krb5kdc[8386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434559, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:55:59.899626 8503 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:55:59.900003 8503 webserver.cc:460] Webserver started at http://127.7.37.193:39115/ using document root <none> and password file <none>
I20260926 14:55:59.900185 8503 fs_manager.cc:359] Metadata directory not provided
I20260926 14:55:59.900233 8503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:55:59.900349 8503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:55:59.901239 8503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "f02ddfb82d3f4294b2f822359e0f4247"
format_stamp: "Formatted at 2026-09-26 14:55:59 on dist-test-slave-f9xb"
server_key: "08c294683a807cda9fe423df19bb2e48"
server_key_iv: "522d65a93ea60031308b732e35781160"
server_key_version: "encryptionkey@0"
I20260926 14:55:59.901530 8503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "f02ddfb82d3f4294b2f822359e0f4247"
format_stamp: "Formatted at 2026-09-26 14:55:59 on dist-test-slave-f9xb"
server_key: "08c294683a807cda9fe423df19bb2e48"
server_key_iv: "522d65a93ea60031308b732e35781160"
server_key_version: "encryptionkey@0"
I20260926 14:55:59.902628 8503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 14:55:59.903191 8518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:59.903330 8503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:55:59.903402 8503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "f02ddfb82d3f4294b2f822359e0f4247"
format_stamp: "Formatted at 2026-09-26 14:55:59 on dist-test-slave-f9xb"
server_key: "08c294683a807cda9fe423df19bb2e48"
server_key_iv: "522d65a93ea60031308b732e35781160"
server_key_version: "encryptionkey@0"
I20260926 14:55:59.903457 8503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:55:59.915848 8503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:55:59.917289 8503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:55:59.917408 8503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:55:59.917805 8503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:55:59.917841 8503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:55:59.917865 8503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:55:59.917882 8503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 14:55:59 dist-test-slave-f9xb krb5kdc[8386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434559, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:55:59.923830 8503 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:33479
I20260926 14:55:59.923872 8633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:33479 every 8 connection(s)
I20260926 14:55:59.924176 8503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:55:59.926160 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:59.918149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:46021 (local address 127.7.37.254:46637)
0926 14:55:59.918209 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:59.918211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:59.918386 (+ 175us) server_negotiation.cc:407] Connection header received
0926 14:55:59.918748 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:59.918750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:59.918764 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:59.918834 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:55:59.919518 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:59.920278 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:59.921234 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:59.921523 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:59.921594 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:59.923762 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:55:59.923763 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:55:59.923763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:55:59.923770 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:55:59.925154 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:59.925465 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:59.925465 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:59.925466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:59.925512 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:59.925678 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:59.925679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:59.925679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:59.925768 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:55:59.925811 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:59.926033 (+ 222us) server_negotiation.cc:299] Negotiation successful
0926 14:55:59.926101 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 14:55:59.926280 8527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:59.918128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:46637 (local address 127.7.37.193:46021)
0926 14:55:59.918272 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 14:55:59.918306 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 14:55:59.918665 (+ 359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:55:59.918898 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:55:59.918901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:55:59.919182 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:55:59.919424 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:59.919429 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:59.920366 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:59.920367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:59.921170 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:59.921172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:59.921620 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:59.921621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:59.921760 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:59.922258 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:55:59.922268 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:55:59.923659 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:55:59.925215 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:55:59.925220 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:55:59.925221 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:55:59.925408 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:55:59.925557 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:55:59.925558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:55:59.925558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:55:59.925635 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:55:59.925810 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:55:59.925812 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:55:59.925909 (+ 97us) client_negotiation.cc:769] Sending connection context
0926 14:55:59.926016 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 14:55:59.926119 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260926 14:55:59.926705 8634 heartbeater.cc:347] Connected to a master server at 127.7.37.254:46637
I20260926 14:55:59.926779 8634 heartbeater.cc:464] Registering TS with master...
I20260926 14:55:59.926937 8634 heartbeater.cc:511] Master 127.7.37.254:46637 requested a full tablet report, sending...
I20260926 14:55:59.927388 8431 ts_manager.cc:194] Registered new tserver with Master: f02ddfb82d3f4294b2f822359e0f4247 (127.7.37.193:33479)
I20260926 14:55:59.927475 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 8503
I20260926 14:55:59.927559 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:55:59.927714 7319 external_mini_cluster.cc:1428] Setting key 22e8be4210aa56f0b5ce09f533910462
I20260926 14:55:59.928273 8431 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:46021
I20260926 14:55:59.932996 8640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:59.927970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:46637 (local address 127.7.37.193:55129)
0926 14:55:59.928076 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0926 14:55:59.928084 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:55:59.928129 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:55:59.928232 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:55:59.928232 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:55:59.928411 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:55:59.928481 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:59.928483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:59.929081 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:59.929082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:59.929759 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:55:59.929762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:55:59.930161 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:55:59.930161 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:55:59.930213 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:59.930557 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:55:59.930561 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:55:59.930800 (+ 239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:55:59.932256 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:55:59.932256 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:55:59.932257 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:55:59.932397 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:55:59.932568 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:55:59.932568 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:55:59.932569 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:55:59.932608 (+ 39us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:55:59.932807 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:55:59.932807 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:55:59.932837 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 14:55:59.932878 (+ 41us) client_negotiation.cc:240] Negotiation successful
0926 14:55:59.932941 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":53,"threads_started":1}
I20260926 14:55:59.933050 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:55:59.927983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:55129 (local address 127.7.37.254:46637)
0926 14:55:59.928014 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 14:55:59.928015 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:55:59.928131 (+ 116us) server_negotiation.cc:407] Connection header received
0926 14:55:59.928169 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:55:59.928170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:55:59.928178 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:55:59.928228 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:55:59.928551 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:59.929014 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:59.929829 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:55:59.930100 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:55:59.930160 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:55:59.930873 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:55:59.930874 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:55:59.930874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:55:59.930880 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:55:59.932189 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:59.932460 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:59.932461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:59.932462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:59.932515 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:55:59.932643 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:55:59.932644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:55:59.932644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:55:59.932754 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:55:59.932812 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 14:55:59.932903 (+ 91us) server_negotiation.cc:299] Negotiation successful
0926 14:55:59.932970 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
WARNING: no policy specified for kudu/127.7.37.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
I20260926 14:55:59.950055 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:46637
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:37587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:00.025130 8643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:00.025296 8643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:00.025318 8643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:00.026496 8643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:00.026533 8643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:00.026552 8643 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 14:56:00.026567 8643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:56:00.027910 8643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:37587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.37.254:46637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:00.028077 8643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:00.028234 8643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:00.032541 8649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:00.032591 8651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:00.032675 8648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:00.032682 8643 server_base.cc:1034] running on GCE node
I20260926 14:56:00.032896 8643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:00.033099 8643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:00.034235 8643 hybrid_clock.cc:648] HybridClock initialized: now 1790434560034220 us; error 26 us; skew 500 ppm
Sep 26 14:56:00 dist-test-slave-f9xb krb5kdc[8386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434560, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:00.036314 8643 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:56:00.036681 8643 webserver.cc:460] Webserver started at http://127.7.37.194:44467/ using document root <none> and password file <none>
I20260926 14:56:00.036871 8643 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:00.036914 8643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:00.037032 8643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:00.037922 8643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "bbd48da2238e479ba63c3c42fcc143ff"
format_stamp: "Formatted at 2026-09-26 14:56:00 on dist-test-slave-f9xb"
server_key: "73e1db4d18eaa02a4d23506a21fa3fe4"
server_key_iv: "59f48ae3d3a2f027d5c11e4abcbdb58a"
server_key_version: "encryptionkey@0"
I20260926 14:56:00.038218 8643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "bbd48da2238e479ba63c3c42fcc143ff"
format_stamp: "Formatted at 2026-09-26 14:56:00 on dist-test-slave-f9xb"
server_key: "73e1db4d18eaa02a4d23506a21fa3fe4"
server_key_iv: "59f48ae3d3a2f027d5c11e4abcbdb58a"
server_key_version: "encryptionkey@0"
I20260926 14:56:00.039299 8643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:00.039891 8658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:00.040024 8643 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:00.040097 8643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "bbd48da2238e479ba63c3c42fcc143ff"
format_stamp: "Formatted at 2026-09-26 14:56:00 on dist-test-slave-f9xb"
server_key: "73e1db4d18eaa02a4d23506a21fa3fe4"
server_key_iv: "59f48ae3d3a2f027d5c11e4abcbdb58a"
server_key_version: "encryptionkey@0"
I20260926 14:56:00.040158 8643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:00.047817 8643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:00.049364 8643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:00.049476 8643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:00.049922 8643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:00.049968 8643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:00.049991 8643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:00.050024 8643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 14:56:00 dist-test-slave-f9xb krb5kdc[8386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434560, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:00.056166 8643 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:37893
I20260926 14:56:00.056227 8773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:37893 every 8 connection(s)
I20260926 14:56:00.056488 8643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:56:00.058305 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.050239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:41689 (local address 127.7.37.254:46637)
0926 14:56:00.050294 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:00.050296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:00.050470 (+ 174us) server_negotiation.cc:407] Connection header received
0926 14:56:00.050881 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:00.050883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:00.050895 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:00.050964 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:00.051607 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.052364 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.053439 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.053650 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.053705 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.056017 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:00.056018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:00.056019 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:00.056026 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:00.057381 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:00.057709 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:00.057709 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:00.057710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:00.057751 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:00.057896 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:00.057897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:00.057897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:00.057982 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:00.058024 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:00.058189 (+ 165us) server_negotiation.cc:299] Negotiation successful
0926 14:56:00.058245 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 14:56:00.058425 8667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.050235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:46637 (local address 127.7.37.194:41689)
0926 14:56:00.050367 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:00.050402 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:00.050795 (+ 393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:00.051022 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:00.051025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:00.051301 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:00.051531 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.051536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.052452 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.052453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.053366 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.053369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.053732 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.053733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.053844 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.054256 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:00.054264 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:00.055935 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:00.057443 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:00.057448 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:00.057449 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:00.057654 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:00.057786 (+ 132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:00.057786 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:00.057786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:00.057860 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:00.058016 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:00.058019 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:00.058111 (+ 92us) client_negotiation.cc:769] Sending connection context
0926 14:56:00.058177 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 14:56:00.058275 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":74,"threads_started":1}
I20260926 14:56:00.058825 8774 heartbeater.cc:347] Connected to a master server at 127.7.37.254:46637
I20260926 14:56:00.058899 8774 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:00.059065 8774 heartbeater.cc:511] Master 127.7.37.254:46637 requested a full tablet report, sending...
I20260926 14:56:00.059376 8430 ts_manager.cc:194] Registered new tserver with Master: bbd48da2238e479ba63c3c42fcc143ff (127.7.37.194:37893)
I20260926 14:56:00.059862 8430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:41689
I20260926 14:56:00.063414 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 8643
I20260926 14:56:00.063494 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:56:00.063649 7319 external_mini_cluster.cc:1428] Setting key 59cbf16732c08a0067097a400bd015ce
I20260926 14:56:00.064236 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.059679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:37667 (local address 127.7.37.254:46637)
0926 14:56:00.059711 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:00.059713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:00.059847 (+ 134us) server_negotiation.cc:407] Connection header received
0926 14:56:00.059901 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:00.059901 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:00.059911 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:00.059957 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:00.060291 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.060740 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.061391 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.061577 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.061626 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.062306 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:00.062307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:00.062308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:00.062314 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:00.063198 (+ 884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:00.063513 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:00.063514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:00.063514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:00.063566 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:00.063719 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:00.063720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:00.063720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:00.063882 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:00.063946 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:00.064089 (+ 143us) server_negotiation.cc:299] Negotiation successful
0926 14:56:00.064151 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260926 14:56:00.064245 8780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.059624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:46637 (local address 127.7.37.194:37667)
0926 14:56:00.059787 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:00.059796 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:00.059849 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:00.059984 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:00.059985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:00.060176 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:00.060226 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.060228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.060819 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.060820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.061319 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.061321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.061630 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.061631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.061700 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.062004 (+ 304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:00.062007 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:00.062236 (+ 229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:00.063281 (+ 1045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:00.063282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:00.063283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:00.063427 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:00.063613 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:00.063614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:00.063614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:00.063668 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:00.063940 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:00.063941 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:00.063985 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 14:56:00.064072 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 14:56:00.064163 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":64,"threads_started":1}
WARNING: no policy specified for kudu/127.7.37.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
I20260926 14:56:00.084204 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:46637
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:37587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:00.158242 8783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:00.158438 8783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:00.158464 8783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:00.159652 8783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:00.159691 8783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:00.159711 8783 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 14:56:00.159726 8783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:56:00.161069 8783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:37587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.37.254:46637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:00.161250 8783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:00.161435 8783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:00.165714 8789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:00.165772 8788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:00.165860 8783 server_base.cc:1034] running on GCE node
W20260926 14:56:00.165915 8791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:00.166082 8783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:00.166302 8783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:00.167448 8783 hybrid_clock.cc:648] HybridClock initialized: now 1790434560167435 us; error 28 us; skew 500 ppm
Sep 26 14:56:00 dist-test-slave-f9xb krb5kdc[8386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434560, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:00.169638 8783 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:56:00.170019 8783 webserver.cc:460] Webserver started at http://127.7.37.195:45283/ using document root <none> and password file <none>
I20260926 14:56:00.170197 8783 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:00.170240 8783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:00.170356 8783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:00.171283 8783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "ea025850c67845d2b60aade34eafdb63"
format_stamp: "Formatted at 2026-09-26 14:56:00 on dist-test-slave-f9xb"
server_key: "f26e0544a14d5259e2615002039ebabf"
server_key_iv: "3bbbf7cf5a79035def4cec13dd138cfe"
server_key_version: "encryptionkey@0"
I20260926 14:56:00.171582 8783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "ea025850c67845d2b60aade34eafdb63"
format_stamp: "Formatted at 2026-09-26 14:56:00 on dist-test-slave-f9xb"
server_key: "f26e0544a14d5259e2615002039ebabf"
server_key_iv: "3bbbf7cf5a79035def4cec13dd138cfe"
server_key_version: "encryptionkey@0"
I20260926 14:56:00.172729 8783 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 14:56:00.173342 8798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:00.173480 8783 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:00.173579 8783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "ea025850c67845d2b60aade34eafdb63"
format_stamp: "Formatted at 2026-09-26 14:56:00 on dist-test-slave-f9xb"
server_key: "f26e0544a14d5259e2615002039ebabf"
server_key_iv: "3bbbf7cf5a79035def4cec13dd138cfe"
server_key_version: "encryptionkey@0"
I20260926 14:56:00.173636 8783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:00.180944 8783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:00.182754 8783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:00.182869 8783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:00.183292 8783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:00.183329 8783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:00.183353 8783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:00.183383 8783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 14:56:00 dist-test-slave-f9xb krb5kdc[8386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434560, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:00.189735 8783 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:33283
I20260926 14:56:00.189800 8913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:33283 every 8 connection(s)
I20260926 14:56:00.190068 8783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:56:00.191995 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.183601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:36695 (local address 127.7.37.254:46637)
0926 14:56:00.183656 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:00.183658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:00.183911 (+ 253us) server_negotiation.cc:407] Connection header received
0926 14:56:00.184277 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:00.184279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:00.184292 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:00.184363 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:00.184876 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.185612 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.186613 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.186898 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.186971 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.189592 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:00.189594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:00.189594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:00.189601 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:00.190932 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:00.191291 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:00.191291 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:00.191292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:00.191338 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:00.191486 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:00.191487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:00.191487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:00.191572 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:00.191613 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:00.191877 (+ 264us) server_negotiation.cc:299] Negotiation successful
0926 14:56:00.191934 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 14:56:00.192114 8807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.183584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:46637 (local address 127.7.37.195:36695)
0926 14:56:00.183795 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:00.183829 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:00.184193 (+ 364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:00.184407 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:00.184410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:00.184610 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:00.184789 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.184793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.185697 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.185698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.186545 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.186547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.186978 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.186979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.187099 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.187557 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:00.187567 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:00.189511 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:00.191015 (+ 1504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:00.191020 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:00.191021 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:00.191233 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:00.191376 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:00.191377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:00.191377 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:00.191454 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:00.191632 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:00.191636 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:00.191738 (+ 102us) client_negotiation.cc:769] Sending connection context
0926 14:56:00.191859 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 14:56:00.191963 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":135,"threads_started":1}
I20260926 14:56:00.192499 8914 heartbeater.cc:347] Connected to a master server at 127.7.37.254:46637
I20260926 14:56:00.192572 8914 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:00.192726 8914 heartbeater.cc:511] Master 127.7.37.254:46637 requested a full tablet report, sending...
I20260926 14:56:00.193015 8431 ts_manager.cc:194] Registered new tserver with Master: ea025850c67845d2b60aade34eafdb63 (127.7.37.195:33283)
I20260926 14:56:00.193372 8431 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:36695
I20260926 14:56:00.197520 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 8783
I20260926 14:56:00.197639 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:56:00.197786 7319 external_mini_cluster.cc:1428] Setting key d8442f6e8b677873c84b7a2829b49095
I20260926 14:56:00.198186 8920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.193353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:46637 (local address 127.7.37.195:50493)
0926 14:56:00.193482 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:00.193492 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:00.193550 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:00.193684 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:00.193685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:00.193861 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:00.193909 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.193911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.194508 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.194509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.194997 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.194999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.195324 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.195325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.195380 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.195761 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:00.195765 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:00.196011 (+ 246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:00.197288 (+ 1277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:00.197288 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:00.197289 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:00.197479 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:00.197647 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:00.197648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:00.197649 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:00.197706 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:00.197896 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:00.197897 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:00.197939 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 14:56:00.198005 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 14:56:00.198101 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":50,"threads_started":1}
I20260926 14:56:00.198194 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.193426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:50493 (local address 127.7.37.254:46637)
0926 14:56:00.193464 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:00.193465 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:00.193538 (+ 73us) server_negotiation.cc:407] Connection header received
0926 14:56:00.193608 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:00.193609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:00.193618 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:00.193666 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:00.193994 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.194428 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.195080 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.195264 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.195319 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.196093 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:00.196094 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:00.196094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:00.196100 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:00.197228 (+ 1128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:00.197543 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:00.197544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:00.197544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:00.197589 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:00.197753 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:00.197753 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:00.197754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:00.197843 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:00.197899 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:00.198032 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 14:56:00.198112 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:56:00.198552 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 14:56:00 dist-test-slave-f9xb krb5kdc[8386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434560, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 14:56:00 dist-test-slave-f9xb krb5kdc[8386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434560, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:00.217969 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.212452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42300 (local address 127.7.37.254:46637)
0926 14:56:00.212508 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:00.212510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:00.212517 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:00.212596 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:00.212597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:00.212603 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:00.212644 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:00.213103 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.213545 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.214177 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.214335 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.214370 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.216167 (+ 1797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:00.216168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:00.216168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:00.216173 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:00.217274 (+ 1101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:00.217516 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:00.217516 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:00.217516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:00.217556 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:00.217666 (+ 110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:00.217666 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:00.217666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:00.217739 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:00.217799 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:00.217871 (+ 72us) server_negotiation.cc:299] Negotiation successful
0926 14:56:00.217917 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 14:56:00.219159 8431 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:56:00.220672 8431 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:56:00.226202 8930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.222357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:37893 (local address 127.0.0.1:36386)
0926 14:56:00.222900 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:00.222908 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:00.222952 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:00.223107 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:00.223107 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:00.223109 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:00.223139 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.223140 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.224386 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.224387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.225171 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.225173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.225930 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.225931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.226011 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.226024 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 14:56:00.226090 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 14:56:00.226135 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"spinlock_wait_cycles":768,"thread_start_us":50,"threads_started":1}
I20260926 14:56:00.226251 8933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.222590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36386 (local address 127.7.37.194:37893)
0926 14:56:00.222717 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:00.222721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:00.222964 (+ 243us) server_negotiation.cc:407] Connection header received
0926 14:56:00.223018 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:00.223020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:00.223034 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:00.223103 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:00.223193 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.224325 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.225264 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.225861 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.225923 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.225978 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:00.226108 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 14:56:00.226149 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260926 14:56:00.226589 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.222388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:33283 (local address 127.0.0.1:55744)
0926 14:56:00.222732 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:00.222764 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:00.222823 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:00.223031 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:00.223034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:00.223036 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:00.223082 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.223083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.224083 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.224083 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.225405 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.225406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.226344 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.226345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.226406 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.226411 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 14:56:00.226449 (+ 38us) client_negotiation.cc:240] Negotiation successful
0926 14:56:00.226489 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":31,"thread_start_us":68,"threads_started":1}
I20260926 14:56:00.226692 8931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.222444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55744 (local address 127.7.37.195:33283)
0926 14:56:00.222547 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:00.222551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:00.222824 (+ 273us) server_negotiation.cc:407] Connection header received
0926 14:56:00.222895 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:00.222897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:00.222911 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:00.222978 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:00.223152 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.224003 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.225485 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.226181 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.226508 (+ 327us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.226564 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:00.226585 (+ 21us) server_negotiation.cc:299] Negotiation successful
0926 14:56:00.226614 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":55,"threads_started":1}
I20260926 14:56:00.226727 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.222569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38772 (local address 127.7.37.193:33479)
0926 14:56:00.222707 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:00.222711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:00.222816 (+ 105us) server_negotiation.cc:407] Connection header received
0926 14:56:00.222891 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:00.222893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:00.222907 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:00.222969 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:00.223185 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.224269 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.225488 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.226218 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.226268 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.226324 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:00.226553 (+ 229us) server_negotiation.cc:299] Negotiation successful
0926 14:56:00.226603 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260926 14:56:00.226854 8929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.222358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:33479 (local address 127.0.0.1:38772)
0926 14:56:00.222730 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:00.222762 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:00.222822 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:00.223036 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:00.223037 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:00.223039 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:00.223103 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.223105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.224343 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.224344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.225402 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.225405 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.226349 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.226350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.226406 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.226412 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 14:56:00.226455 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 14:56:00.226489 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":6,"thread_start_us":50,"threads_started":1}
I20260926 14:56:00.226936 8708 tablet_service.cc:1467] Processing CreateTablet for tablet b0c2b730d1714e9ba14b3bb418025a68 (DEFAULT_TABLE table=test-table [id=c04ac6b5a6b041198d0ae4c7ef46a363]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:00.227257 8708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0c2b730d1714e9ba14b3bb418025a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:00.227244 8848 tablet_service.cc:1467] Processing CreateTablet for tablet b0c2b730d1714e9ba14b3bb418025a68 (DEFAULT_TABLE table=test-table [id=c04ac6b5a6b041198d0ae4c7ef46a363]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:00.227494 8848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0c2b730d1714e9ba14b3bb418025a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:00.229611 8934 tablet_bootstrap.cc:492] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff: Bootstrap starting.
I20260926 14:56:00.229671 8935 tablet_bootstrap.cc:492] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63: Bootstrap starting.
I20260926 14:56:00.229945 8568 tablet_service.cc:1467] Processing CreateTablet for tablet b0c2b730d1714e9ba14b3bb418025a68 (DEFAULT_TABLE table=test-table [id=c04ac6b5a6b041198d0ae4c7ef46a363]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:00.230258 8568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0c2b730d1714e9ba14b3bb418025a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:00.230345 8935 tablet_bootstrap.cc:654] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:00.230575 8934 tablet_bootstrap.cc:654] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:00.230660 8935 log.cc:824] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:00.230872 8934 log.cc:824] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:00.231323 8935 tablet_bootstrap.cc:492] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63: No bootstrap required, opened a new log
I20260926 14:56:00.231379 8935 ts_tablet_manager.cc:1397] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:56:00.231544 8934 tablet_bootstrap.cc:492] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff: No bootstrap required, opened a new log
I20260926 14:56:00.231608 8934 ts_tablet_manager.cc:1397] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:56:00.232786 8935 raft_consensus.cc:348] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.232903 8935 raft_consensus.cc:374] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:00.232929 8935 raft_consensus.cc:729] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea025850c67845d2b60aade34eafdb63, State: Initialized, Role: FOLLOWER
I20260926 14:56:00.233047 8935 consensus_queue.cc:260] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.233271 8935 ts_tablet_manager.cc:1428] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:56:00.233377 8914 heartbeater.cc:503] Master 127.7.37.254:46637 was elected leader, sending a full tablet report...
I20260926 14:56:00.233597 8934 raft_consensus.cc:348] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.233744 8934 raft_consensus.cc:374] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:00.233778 8934 raft_consensus.cc:729] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbd48da2238e479ba63c3c42fcc143ff, State: Initialized, Role: FOLLOWER
I20260926 14:56:00.233872 8934 consensus_queue.cc:260] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.234011 8939 tablet_bootstrap.cc:492] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247: Bootstrap starting.
I20260926 14:56:00.234113 8934 ts_tablet_manager.cc:1428] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 14:56:00.234254 8774 heartbeater.cc:503] Master 127.7.37.254:46637 was elected leader, sending a full tablet report...
I20260926 14:56:00.234549 8939 tablet_bootstrap.cc:654] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:00.234827 8939 log.cc:824] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:00.235357 8939 tablet_bootstrap.cc:492] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247: No bootstrap required, opened a new log
I20260926 14:56:00.235409 8939 ts_tablet_manager.cc:1397] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:00.236673 8939 raft_consensus.cc:348] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.236779 8939 raft_consensus.cc:374] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:00.236809 8939 raft_consensus.cc:729] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f02ddfb82d3f4294b2f822359e0f4247, State: Initialized, Role: FOLLOWER
I20260926 14:56:00.236919 8939 consensus_queue.cc:260] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.237145 8939 ts_tablet_manager.cc:1428] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 14:56:00.237161 8634 heartbeater.cc:503] Master 127.7.37.254:46637 was elected leader, sending a full tablet report...
I20260926 14:56:00.296073 8942 raft_consensus.cc:482] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:56:00.296149 8942 raft_consensus.cc:504] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.296384 8942 leader_election.cc:288] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbd48da2238e479ba63c3c42fcc143ff (127.7.37.194:37893), ea025850c67845d2b60aade34eafdb63 (127.7.37.195:33283)
I20260926 14:56:00.298775 8527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.296524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:37893 (local address 127.7.37.193:38015)
0926 14:56:00.296600 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:00.296611 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:00.296650 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:00.296766 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:00.296767 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:00.296768 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:00.296818 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.296819 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.297397 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.297398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.298089 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.298091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.298566 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.298567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.298628 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.298631 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:00.298673 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 14:56:00.298707 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260926 14:56:00.298775 8933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.296605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:38015 (local address 127.7.37.194:37893)
0926 14:56:00.296643 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:00.296645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:00.296653 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:00.296701 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:00.296702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:00.296711 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:00.296754 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:00.296870 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.297326 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.298166 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.298491 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.298547 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.298554 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:00.298687 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 14:56:00.298720 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 14:56:00.298869 8931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.296622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:56635 (local address 127.7.37.195:33283)
0926 14:56:00.296658 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:00.296660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:00.296749 (+ 89us) server_negotiation.cc:407] Connection header received
0926 14:56:00.296796 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:00.296797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:00.296806 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:00.296845 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:00.296965 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.297399 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.298242 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.298582 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.298639 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.298646 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:00.298782 (+ 136us) server_negotiation.cc:299] Negotiation successful
0926 14:56:00.298813 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 14:56:00.299048 8728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0c2b730d1714e9ba14b3bb418025a68" candidate_uuid: "f02ddfb82d3f4294b2f822359e0f4247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbd48da2238e479ba63c3c42fcc143ff" is_pre_election: true
I20260926 14:56:00.299167 8728 raft_consensus.cc:2454] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f02ddfb82d3f4294b2f822359e0f4247 in term 0.
I20260926 14:56:00.299366 8943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.296537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:33283 (local address 127.7.37.193:56635)
0926 14:56:00.296689 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:00.296698 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:00.296752 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:00.296858 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:00.296859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:00.296860 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:00.296918 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.296920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.297468 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.297469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.298171 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.298172 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.298658 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.298659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.298729 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.298732 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:00.298773 (+ 41us) client_negotiation.cc:240] Negotiation successful
0926 14:56:00.298808 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":77,"threads_started":1}
I20260926 14:56:00.299383 8520 leader_election.cc:302] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbd48da2238e479ba63c3c42fcc143ff, f02ddfb82d3f4294b2f822359e0f4247; no voters:
I20260926 14:56:00.299510 8942 raft_consensus.cc:2781] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:56:00.299556 8942 raft_consensus.cc:482] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:56:00.299582 8942 raft_consensus.cc:3037] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:00.299634 8868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0c2b730d1714e9ba14b3bb418025a68" candidate_uuid: "f02ddfb82d3f4294b2f822359e0f4247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea025850c67845d2b60aade34eafdb63" is_pre_election: true
I20260926 14:56:00.299729 8868 raft_consensus.cc:2454] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f02ddfb82d3f4294b2f822359e0f4247 in term 0.
I20260926 14:56:00.300381 8942 raft_consensus.cc:504] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.300493 8942 leader_election.cc:288] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [CANDIDATE]: Term 1 election: Requested vote from peers bbd48da2238e479ba63c3c42fcc143ff (127.7.37.194:37893), ea025850c67845d2b60aade34eafdb63 (127.7.37.195:33283)
I20260926 14:56:00.300638 8868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0c2b730d1714e9ba14b3bb418025a68" candidate_uuid: "f02ddfb82d3f4294b2f822359e0f4247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea025850c67845d2b60aade34eafdb63"
I20260926 14:56:00.300634 8728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0c2b730d1714e9ba14b3bb418025a68" candidate_uuid: "f02ddfb82d3f4294b2f822359e0f4247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbd48da2238e479ba63c3c42fcc143ff"
I20260926 14:56:00.300704 8868 raft_consensus.cc:3037] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:00.300711 8728 raft_consensus.cc:3037] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:00.301266 8868 raft_consensus.cc:2454] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f02ddfb82d3f4294b2f822359e0f4247 in term 1.
I20260926 14:56:00.301337 8728 raft_consensus.cc:2454] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f02ddfb82d3f4294b2f822359e0f4247 in term 1.
I20260926 14:56:00.301412 8522 leader_election.cc:302] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea025850c67845d2b60aade34eafdb63, f02ddfb82d3f4294b2f822359e0f4247; no voters:
I20260926 14:56:00.301484 8942 raft_consensus.cc:2781] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:00.301580 8942 raft_consensus.cc:686] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [term 1 LEADER]: Becoming Leader. State: Replica: f02ddfb82d3f4294b2f822359e0f4247, State: Running, Role: LEADER
I20260926 14:56:00.301638 8942 consensus_queue.cc:237] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.302227 8431 catalog_manager.cc:5495] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 reported cstate change: term changed from 0 to 1, leader changed from <none> to f02ddfb82d3f4294b2f822359e0f4247 (127.7.37.193). New cstate: current_term: 1 leader_uuid: "f02ddfb82d3f4294b2f822359e0f4247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } health_report { overall_health: UNKNOWN } } }
W20260926 14:56:00.307017 8775 tablet.cc:2369] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:56:00.317308 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.314705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38778 (local address 127.7.37.193:33479)
0926 14:56:00.314760 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:00.314762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:00.314800 (+ 38us) server_negotiation.cc:407] Connection header received
0926 14:56:00.314902 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:00.314902 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:00.314916 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:00.314971 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:00.315128 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.315785 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.316528 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.316778 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.316831 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.317024 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:00.317114 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:00.317184 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:00.317219 (+ 35us) server_negotiation.cc:299] Negotiation successful
0926 14:56:00.317235 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 14:56:00.318883 8868 raft_consensus.cc:1261] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63 [term 1 FOLLOWER]: Refusing update from remote peer f02ddfb82d3f4294b2f822359e0f4247: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:00.318898 8728 raft_consensus.cc:1261] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff [term 1 FOLLOWER]: Refusing update from remote peer f02ddfb82d3f4294b2f822359e0f4247: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:00.319134 8944 consensus_queue.cc:1035] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:00.319237 8942 consensus_queue.cc:1035] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:00.321048 8947 mvcc.cc:205] Tried to move back new op lower bound from 7333619959063932928 to 7333619958995832832. Current Snapshot: MvccSnapshot[applied={T|T < 7333619959063932928}]
I20260926 14:56:00.321373 8948 mvcc.cc:205] Tried to move back new op lower bound from 7333619959063932928 to 7333619958995832832. Current Snapshot: MvccSnapshot[applied={T|T < 7333619959063932928}]
I20260926 14:56:00.321214 8949 mvcc.cc:205] Tried to move back new op lower bound from 7333619959063932928 to 7333619958995832832. Current Snapshot: MvccSnapshot[applied={T|T < 7333619959063932928}]
I20260926 14:56:00.323921 8431 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:00.323997 8431 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:00.324574 8431 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f: Sending DeleteTablet for 3 replicas of tablet b0c2b730d1714e9ba14b3bb418025a68
I20260926 14:56:00.324859 8568 tablet_service.cc:1514] Processing DeleteTablet for tablet b0c2b730d1714e9ba14b3bb418025a68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:00 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:38772
I20260926 14:56:00.324859 8708 tablet_service.cc:1514] Processing DeleteTablet for tablet b0c2b730d1714e9ba14b3bb418025a68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:00 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:36386
I20260926 14:56:00.325134 8848 tablet_service.cc:1514] Processing DeleteTablet for tablet b0c2b730d1714e9ba14b3bb418025a68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:00 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:55744
I20260926 14:56:00.325129 8957 tablet_replica.cc:330] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247: stopping tablet replica
I20260926 14:56:00.325333 8957 raft_consensus.cc:2229] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:56:00.325354 8959 tablet_replica.cc:330] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63: stopping tablet replica
I20260926 14:56:00.325420 8957 raft_consensus.cc:2258] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:00.325372 8431 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:56:00.325490 8431 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:56:00.325644 8959 raft_consensus.cc:2229] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:00.325791 8959 raft_consensus.cc:2258] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:00.325933 8957 ts_tablet_manager.cc:1905] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:00.326264 8959 ts_tablet_manager.cc:1905] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:00.326536 8568 tablet_service.cc:1467] Processing CreateTablet for tablet 0b6c68642d6a4556961691037ac8d8dd (DEFAULT_TABLE table=test-table [id=4b6eb64f810e4179ac7f189386f2fb77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:00.326639 8568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b6c68642d6a4556961691037ac8d8dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:00.327837 8957 ts_tablet_manager.cc:1918] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:56:00.327903 8957 log.cc:1196] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-0/wal/wals/b0c2b730d1714e9ba14b3bb418025a68
I20260926 14:56:00.328122 8957 ts_tablet_manager.cc:1939] T b0c2b730d1714e9ba14b3bb418025a68 P f02ddfb82d3f4294b2f822359e0f4247: Deleting consensus metadata
I20260926 14:56:00.328249 8939 tablet_bootstrap.cc:492] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247: Bootstrap starting.
I20260926 14:56:00.328436 8415 catalog_manager.cc:4841] TS f02ddfb82d3f4294b2f822359e0f4247 (127.7.37.193:33479): tablet b0c2b730d1714e9ba14b3bb418025a68 (table test-table [id=c04ac6b5a6b041198d0ae4c7ef46a363]) successfully deleted
I20260926 14:56:00.328595 8939 tablet_bootstrap.cc:654] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:00.328575 8848 tablet_service.cc:1467] Processing CreateTablet for tablet 0b6c68642d6a4556961691037ac8d8dd (DEFAULT_TABLE table=test-table [id=4b6eb64f810e4179ac7f189386f2fb77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:00.328732 8708 tablet_service.cc:1467] Processing CreateTablet for tablet 0b6c68642d6a4556961691037ac8d8dd (DEFAULT_TABLE table=test-table [id=4b6eb64f810e4179ac7f189386f2fb77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:00.328831 8708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b6c68642d6a4556961691037ac8d8dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:00.329022 8848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b6c68642d6a4556961691037ac8d8dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:00.329010 8959 ts_tablet_manager.cc:1918] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:56:00.329188 8959 log.cc:1196] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/b0c2b730d1714e9ba14b3bb418025a68
I20260926 14:56:00.329550 8959 ts_tablet_manager.cc:1939] T b0c2b730d1714e9ba14b3bb418025a68 P ea025850c67845d2b60aade34eafdb63: Deleting consensus metadata
I20260926 14:56:00.329928 8418 catalog_manager.cc:4841] TS ea025850c67845d2b60aade34eafdb63 (127.7.37.195:33283): tablet b0c2b730d1714e9ba14b3bb418025a68 (table test-table [id=c04ac6b5a6b041198d0ae4c7ef46a363]) successfully deleted
I20260926 14:56:00.329991 8934 tablet_bootstrap.cc:492] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff: Bootstrap starting.
I20260926 14:56:00.330003 8935 tablet_bootstrap.cc:492] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63: Bootstrap starting.
I20260926 14:56:00.330307 8934 tablet_bootstrap.cc:654] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:00.330327 8935 tablet_bootstrap.cc:654] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:00.330336 8939 tablet_bootstrap.cc:492] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247: No bootstrap required, opened a new log
I20260926 14:56:00.330368 8939 ts_tablet_manager.cc:1397] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 14:56:00.330490 8939 raft_consensus.cc:348] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.330541 8939 raft_consensus.cc:374] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:00.330559 8939 raft_consensus.cc:729] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f02ddfb82d3f4294b2f822359e0f4247, State: Initialized, Role: FOLLOWER
I20260926 14:56:00.328723 8958 tablet_replica.cc:330] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff: stopping tablet replica
I20260926 14:56:00.330593 8939 consensus_queue.cc:260] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.330667 8939 ts_tablet_manager.cc:1428] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:00.330668 8958 raft_consensus.cc:2229] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:00.330788 8958 raft_consensus.cc:2258] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:00.331175 8958 ts_tablet_manager.cc:1905] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:00.331316 8934 tablet_bootstrap.cc:492] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff: No bootstrap required, opened a new log
I20260926 14:56:00.331357 8934 ts_tablet_manager.cc:1397] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:00.331475 8934 raft_consensus.cc:348] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.331542 8934 raft_consensus.cc:374] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:00.331558 8934 raft_consensus.cc:729] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbd48da2238e479ba63c3c42fcc143ff, State: Initialized, Role: FOLLOWER
I20260926 14:56:00.331597 8934 consensus_queue.cc:260] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.331692 8934 ts_tablet_manager.cc:1428] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:00.332437 8958 ts_tablet_manager.cc:1918] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:56:00.332489 8958 log.cc:1196] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/b0c2b730d1714e9ba14b3bb418025a68
I20260926 14:56:00.332629 8958 ts_tablet_manager.cc:1939] T b0c2b730d1714e9ba14b3bb418025a68 P bbd48da2238e479ba63c3c42fcc143ff: Deleting consensus metadata
I20260926 14:56:00.333067 8416 catalog_manager.cc:4841] TS bbd48da2238e479ba63c3c42fcc143ff (127.7.37.194:37893): tablet b0c2b730d1714e9ba14b3bb418025a68 (table test-table [id=c04ac6b5a6b041198d0ae4c7ef46a363]) successfully deleted
I20260926 14:56:00.336058 8935 tablet_bootstrap.cc:492] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63: No bootstrap required, opened a new log
I20260926 14:56:00.336097 8935 ts_tablet_manager.cc:1397] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260926 14:56:00.336210 8935 raft_consensus.cc:348] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.336262 8935 raft_consensus.cc:374] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:00.336274 8935 raft_consensus.cc:729] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea025850c67845d2b60aade34eafdb63, State: Initialized, Role: FOLLOWER
I20260926 14:56:00.336305 8935 consensus_queue.cc:260] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.336388 8935 ts_tablet_manager.cc:1428] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260926 14:56:00.424762 8635 tablet.cc:2369] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 14:56:00.440510 8915 tablet.cc:2369] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:56:00.645332 8942 raft_consensus.cc:482] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:56:00.645409 8942 raft_consensus.cc:504] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.645561 8942 leader_election.cc:288] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbd48da2238e479ba63c3c42fcc143ff (127.7.37.194:37893), ea025850c67845d2b60aade34eafdb63 (127.7.37.195:33283)
I20260926 14:56:00.645750 8728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b6c68642d6a4556961691037ac8d8dd" candidate_uuid: "f02ddfb82d3f4294b2f822359e0f4247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbd48da2238e479ba63c3c42fcc143ff" is_pre_election: true
I20260926 14:56:00.645763 8868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b6c68642d6a4556961691037ac8d8dd" candidate_uuid: "f02ddfb82d3f4294b2f822359e0f4247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea025850c67845d2b60aade34eafdb63" is_pre_election: true
I20260926 14:56:00.645831 8728 raft_consensus.cc:2454] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f02ddfb82d3f4294b2f822359e0f4247 in term 0.
I20260926 14:56:00.645849 8868 raft_consensus.cc:2454] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f02ddfb82d3f4294b2f822359e0f4247 in term 0.
I20260926 14:56:00.645972 8520 leader_election.cc:302] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: bbd48da2238e479ba63c3c42fcc143ff, ea025850c67845d2b60aade34eafdb63, f02ddfb82d3f4294b2f822359e0f4247; no voters:
I20260926 14:56:00.646090 8942 raft_consensus.cc:2781] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:56:00.646122 8942 raft_consensus.cc:482] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:56:00.646135 8942 raft_consensus.cc:3037] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:00.646706 8942 raft_consensus.cc:504] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.646839 8942 leader_election.cc:288] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [CANDIDATE]: Term 1 election: Requested vote from peers bbd48da2238e479ba63c3c42fcc143ff (127.7.37.194:37893), ea025850c67845d2b60aade34eafdb63 (127.7.37.195:33283)
I20260926 14:56:00.646960 8728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b6c68642d6a4556961691037ac8d8dd" candidate_uuid: "f02ddfb82d3f4294b2f822359e0f4247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbd48da2238e479ba63c3c42fcc143ff"
I20260926 14:56:00.646976 8868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b6c68642d6a4556961691037ac8d8dd" candidate_uuid: "f02ddfb82d3f4294b2f822359e0f4247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea025850c67845d2b60aade34eafdb63"
I20260926 14:56:00.647029 8728 raft_consensus.cc:3037] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:00.647038 8868 raft_consensus.cc:3037] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:00.647708 8728 raft_consensus.cc:2454] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f02ddfb82d3f4294b2f822359e0f4247 in term 1.
I20260926 14:56:00.647707 8868 raft_consensus.cc:2454] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f02ddfb82d3f4294b2f822359e0f4247 in term 1.
I20260926 14:56:00.647886 8522 leader_election.cc:302] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea025850c67845d2b60aade34eafdb63, f02ddfb82d3f4294b2f822359e0f4247; no voters:
I20260926 14:56:00.647987 8942 raft_consensus.cc:2781] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:00.648034 8942 raft_consensus.cc:686] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [term 1 LEADER]: Becoming Leader. State: Replica: f02ddfb82d3f4294b2f822359e0f4247, State: Running, Role: LEADER
I20260926 14:56:00.648080 8942 consensus_queue.cc:237] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.648562 8431 catalog_manager.cc:5495] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 reported cstate change: term changed from 0 to 1, leader changed from <none> to f02ddfb82d3f4294b2f822359e0f4247 (127.7.37.193). New cstate: current_term: 1 leader_uuid: "f02ddfb82d3f4294b2f822359e0f4247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } health_report { overall_health: UNKNOWN } } }
Sep 26 14:56:00 dist-test-slave-f9xb krb5kdc[8386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434559, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:00.702243 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.696326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42316 (local address 127.7.37.254:46637)
0926 14:56:00.696414 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:00.696416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:00.696530 (+ 114us) server_negotiation.cc:407] Connection header received
0926 14:56:00.696589 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:00.696589 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:00.696599 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:00.696643 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:00.697043 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.697497 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.698151 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.698317 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.698372 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.700255 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:00.700256 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:00.700256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:00.700262 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:00.701298 (+ 1036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:00.701592 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:00.701593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:00.701594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:00.701643 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:00.701787 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:00.701788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:00.701788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:00.701883 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:00.701934 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:00.702059 (+ 125us) server_negotiation.cc:299] Negotiation successful
0926 14:56:00.702134 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 14:56:00.702315 8969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.696251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:46637 (local address 127.0.0.1:42316)
0926 14:56:00.696463 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:00.696474 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:00.696532 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:00.696651 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:00.696652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:00.696933 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:00.696972 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.696974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.697564 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.697564 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.698084 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.698086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.698369 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.698370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.698431 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.698890 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:00.698894 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:00.700184 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:00.701362 (+ 1178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:00.701369 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:00.701371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:00.701532 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:00.701694 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:00.701694 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:00.701695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:00.701742 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:00.701931 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:00.701937 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:00.701980 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 14:56:00.702035 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 14:56:00.702185 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":57,"thread_start_us":69,"threads_started":1}
I20260926 14:56:00.704118 8431 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:42316:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 14:56:00.704274 8431 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:56:00.705454 8708 tablet_service.cc:1467] Processing CreateTablet for tablet 5b8e3b2f47c94477801e07c4ccb2892c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c27c3986e4154c7cbedc786dc7deeeb9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 14:56:00.705444 8568 tablet_service.cc:1467] Processing CreateTablet for tablet 5b8e3b2f47c94477801e07c4ccb2892c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c27c3986e4154c7cbedc786dc7deeeb9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 14:56:00.705590 8708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b8e3b2f47c94477801e07c4ccb2892c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:00.705596 8568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b8e3b2f47c94477801e07c4ccb2892c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:00.705559 8848 tablet_service.cc:1467] Processing CreateTablet for tablet 5b8e3b2f47c94477801e07c4ccb2892c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c27c3986e4154c7cbedc786dc7deeeb9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 14:56:00.705683 8848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b8e3b2f47c94477801e07c4ccb2892c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:00.706802 8935 tablet_bootstrap.cc:492] T 5b8e3b2f47c94477801e07c4ccb2892c P ea025850c67845d2b60aade34eafdb63: Bootstrap starting.
I20260926 14:56:00.706913 8939 tablet_bootstrap.cc:492] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247: Bootstrap starting.
I20260926 14:56:00.707176 8939 tablet_bootstrap.cc:654] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:00.707176 8935 tablet_bootstrap.cc:654] T 5b8e3b2f47c94477801e07c4ccb2892c P ea025850c67845d2b60aade34eafdb63: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:00.707418 8934 tablet_bootstrap.cc:492] T 5b8e3b2f47c94477801e07c4ccb2892c P bbd48da2238e479ba63c3c42fcc143ff: Bootstrap starting.
I20260926 14:56:00.707641 8939 tablet_bootstrap.cc:492] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247: No bootstrap required, opened a new log
I20260926 14:56:00.707681 8939 ts_tablet_manager.cc:1397] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260926 14:56:00.707803 8934 tablet_bootstrap.cc:654] T 5b8e3b2f47c94477801e07c4ccb2892c P bbd48da2238e479ba63c3c42fcc143ff: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:00.707798 8939 raft_consensus.cc:348] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.707850 8939 raft_consensus.cc:374] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:00.707863 8939 raft_consensus.cc:729] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f02ddfb82d3f4294b2f822359e0f4247, State: Initialized, Role: FOLLOWER
I20260926 14:56:00.707907 8939 consensus_queue.cc:260] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.707989 8939 ts_tablet_manager.cc:1428] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:00.708029 8942 tablet_replica.cc:439] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } } }
I20260926 14:56:00.708109 8942 tablet_replica.cc:442] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:00.708360 8935 tablet_bootstrap.cc:492] T 5b8e3b2f47c94477801e07c4ccb2892c P ea025850c67845d2b60aade34eafdb63: No bootstrap required, opened a new log
I20260926 14:56:00.708403 8935 ts_tablet_manager.cc:1397] T 5b8e3b2f47c94477801e07c4ccb2892c P ea025850c67845d2b60aade34eafdb63: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:56:00.708460 8934 tablet_bootstrap.cc:492] T 5b8e3b2f47c94477801e07c4ccb2892c P bbd48da2238e479ba63c3c42fcc143ff: No bootstrap required, opened a new log
I20260926 14:56:00.708511 8934 ts_tablet_manager.cc:1397] T 5b8e3b2f47c94477801e07c4ccb2892c P bbd48da2238e479ba63c3c42fcc143ff: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 14:56:00.708537 8935 raft_consensus.cc:348] T 5b8e3b2f47c94477801e07c4ccb2892c P ea025850c67845d2b60aade34eafdb63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.708598 8935 raft_consensus.cc:374] T 5b8e3b2f47c94477801e07c4ccb2892c P ea025850c67845d2b60aade34eafdb63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:00.708621 8935 raft_consensus.cc:729] T 5b8e3b2f47c94477801e07c4ccb2892c P ea025850c67845d2b60aade34eafdb63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea025850c67845d2b60aade34eafdb63, State: Initialized, Role: FOLLOWER
I20260926 14:56:00.708657 8935 consensus_queue.cc:260] T 5b8e3b2f47c94477801e07c4ccb2892c P ea025850c67845d2b60aade34eafdb63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.708658 8934 raft_consensus.cc:348] T 5b8e3b2f47c94477801e07c4ccb2892c P bbd48da2238e479ba63c3c42fcc143ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.708724 8934 raft_consensus.cc:374] T 5b8e3b2f47c94477801e07c4ccb2892c P bbd48da2238e479ba63c3c42fcc143ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:00.708732 8935 ts_tablet_manager.cc:1428] T 5b8e3b2f47c94477801e07c4ccb2892c P ea025850c67845d2b60aade34eafdb63: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:00.708747 8934 raft_consensus.cc:729] T 5b8e3b2f47c94477801e07c4ccb2892c P bbd48da2238e479ba63c3c42fcc143ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbd48da2238e479ba63c3c42fcc143ff, State: Initialized, Role: FOLLOWER
I20260926 14:56:00.708796 8934 consensus_queue.cc:260] T 5b8e3b2f47c94477801e07c4ccb2892c P bbd48da2238e479ba63c3c42fcc143ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.708878 8934 ts_tablet_manager.cc:1428] T 5b8e3b2f47c94477801e07c4ccb2892c P bbd48da2238e479ba63c3c42fcc143ff: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:00.708851 8938 tablet_replica.cc:439] T 5b8e3b2f47c94477801e07c4ccb2892c P ea025850c67845d2b60aade34eafdb63: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } } }
I20260926 14:56:00.708909 8938 tablet_replica.cc:442] T 5b8e3b2f47c94477801e07c4ccb2892c P ea025850c67845d2b60aade34eafdb63: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:00.708918 8940 tablet_replica.cc:439] T 5b8e3b2f47c94477801e07c4ccb2892c P bbd48da2238e479ba63c3c42fcc143ff: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } } }
I20260926 14:56:00.708997 8940 tablet_replica.cc:442] T 5b8e3b2f47c94477801e07c4ccb2892c P bbd48da2238e479ba63c3c42fcc143ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:00.743273 8942 raft_consensus.cc:482] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:56:00.743332 8942 raft_consensus.cc:504] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.743428 8942 leader_election.cc:288] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbd48da2238e479ba63c3c42fcc143ff (127.7.37.194:37893), ea025850c67845d2b60aade34eafdb63 (127.7.37.195:33283)
I20260926 14:56:00.743548 8728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b8e3b2f47c94477801e07c4ccb2892c" candidate_uuid: "f02ddfb82d3f4294b2f822359e0f4247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbd48da2238e479ba63c3c42fcc143ff" is_pre_election: true
I20260926 14:56:00.743642 8728 raft_consensus.cc:2454] T 5b8e3b2f47c94477801e07c4ccb2892c P bbd48da2238e479ba63c3c42fcc143ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f02ddfb82d3f4294b2f822359e0f4247 in term 0.
I20260926 14:56:00.743623 8868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b8e3b2f47c94477801e07c4ccb2892c" candidate_uuid: "f02ddfb82d3f4294b2f822359e0f4247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea025850c67845d2b60aade34eafdb63" is_pre_election: true
I20260926 14:56:00.743702 8868 raft_consensus.cc:2454] T 5b8e3b2f47c94477801e07c4ccb2892c P ea025850c67845d2b60aade34eafdb63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f02ddfb82d3f4294b2f822359e0f4247 in term 0.
I20260926 14:56:00.743786 8520 leader_election.cc:302] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbd48da2238e479ba63c3c42fcc143ff, f02ddfb82d3f4294b2f822359e0f4247; no voters:
I20260926 14:56:00.743875 8942 raft_consensus.cc:2781] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:56:00.743920 8942 raft_consensus.cc:482] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:56:00.743939 8942 raft_consensus.cc:3037] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:00.744498 8942 raft_consensus.cc:504] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.744606 8942 leader_election.cc:288] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 [CANDIDATE]: Term 1 election: Requested vote from peers bbd48da2238e479ba63c3c42fcc143ff (127.7.37.194:37893), ea025850c67845d2b60aade34eafdb63 (127.7.37.195:33283)
I20260926 14:56:00.744711 8868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b8e3b2f47c94477801e07c4ccb2892c" candidate_uuid: "f02ddfb82d3f4294b2f822359e0f4247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea025850c67845d2b60aade34eafdb63"
I20260926 14:56:00.744769 8868 raft_consensus.cc:3037] T 5b8e3b2f47c94477801e07c4ccb2892c P ea025850c67845d2b60aade34eafdb63 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:00.744756 8728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b8e3b2f47c94477801e07c4ccb2892c" candidate_uuid: "f02ddfb82d3f4294b2f822359e0f4247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbd48da2238e479ba63c3c42fcc143ff"
I20260926 14:56:00.744804 8728 raft_consensus.cc:3037] T 5b8e3b2f47c94477801e07c4ccb2892c P bbd48da2238e479ba63c3c42fcc143ff [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:00.745427 8728 raft_consensus.cc:2454] T 5b8e3b2f47c94477801e07c4ccb2892c P bbd48da2238e479ba63c3c42fcc143ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f02ddfb82d3f4294b2f822359e0f4247 in term 1.
I20260926 14:56:00.745427 8868 raft_consensus.cc:2454] T 5b8e3b2f47c94477801e07c4ccb2892c P ea025850c67845d2b60aade34eafdb63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f02ddfb82d3f4294b2f822359e0f4247 in term 1.
I20260926 14:56:00.745601 8520 leader_election.cc:302] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbd48da2238e479ba63c3c42fcc143ff, f02ddfb82d3f4294b2f822359e0f4247; no voters:
I20260926 14:56:00.745698 8942 raft_consensus.cc:2781] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:00.745761 8942 raft_consensus.cc:686] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 [term 1 LEADER]: Becoming Leader. State: Replica: f02ddfb82d3f4294b2f822359e0f4247, State: Running, Role: LEADER
I20260926 14:56:00.745808 8942 consensus_queue.cc:237] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } }
I20260926 14:56:00.746042 8952 tablet_replica.cc:439] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247: TxnStatusTablet state changed. Reason: New leader f02ddfb82d3f4294b2f822359e0f4247. Latest consensus state: current_term: 1 leader_uuid: "f02ddfb82d3f4294b2f822359e0f4247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } } }
I20260926 14:56:00.746115 8952 tablet_replica.cc:442] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247: This TxnStatusTablet replica's current role is: LEADER
I20260926 14:56:00.746248 8974 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 14:56:00.746244 8430 catalog_manager.cc:5495] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 reported cstate change: term changed from 0 to 1, leader changed from <none> to f02ddfb82d3f4294b2f822359e0f4247 (127.7.37.193). New cstate: current_term: 1 leader_uuid: "f02ddfb82d3f4294b2f822359e0f4247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } health_report { overall_health: UNKNOWN } } }
I20260926 14:56:00.746337 8974 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 14:56:00.756605 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.754512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36390 (local address 127.7.37.193:33479)
0926 14:56:00.754560 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:00.754562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:00.754571 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:56:00.754635 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:00.754636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:00.754645 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:00.754695 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:00.754797 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.755254 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.756005 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.756198 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.756248 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.756389 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:00.756423 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:00.756463 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:00.756515 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 14:56:00.756535 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 14:56:00.756615 8969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.754448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:33479 (local address 127.0.0.1:36390)
0926 14:56:00.754518 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:00.754529 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:00.754576 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:00.754702 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:00.754704 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:00.754705 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 14:56:00.754748 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.754750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.755318 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.755319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.755933 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.755935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.756251 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.756252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.756314 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.756327 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 14:56:00.756468 (+ 141us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 14:56:00.756475 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 14:56:00.756513 (+ 38us) client_negotiation.cc:240] Negotiation successful
0926 14:56:00.756556 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 14:56:00.757362 8868 raft_consensus.cc:1261] T 5b8e3b2f47c94477801e07c4ccb2892c P ea025850c67845d2b60aade34eafdb63 [term 1 FOLLOWER]: Refusing update from remote peer f02ddfb82d3f4294b2f822359e0f4247: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:00.757391 8728 raft_consensus.cc:1261] T 5b8e3b2f47c94477801e07c4ccb2892c P bbd48da2238e479ba63c3c42fcc143ff [term 1 FOLLOWER]: Refusing update from remote peer f02ddfb82d3f4294b2f822359e0f4247: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:00.757550 8942 consensus_queue.cc:1035] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:00.757645 8952 consensus_queue.cc:1035] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:00.758087 8940 tablet_replica.cc:439] T 5b8e3b2f47c94477801e07c4ccb2892c P bbd48da2238e479ba63c3c42fcc143ff: TxnStatusTablet state changed. Reason: New leader f02ddfb82d3f4294b2f822359e0f4247. Latest consensus state: current_term: 1 leader_uuid: "f02ddfb82d3f4294b2f822359e0f4247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } } }
I20260926 14:56:00.758091 8938 tablet_replica.cc:439] T 5b8e3b2f47c94477801e07c4ccb2892c P ea025850c67845d2b60aade34eafdb63: TxnStatusTablet state changed. Reason: New leader f02ddfb82d3f4294b2f822359e0f4247. Latest consensus state: current_term: 1 leader_uuid: "f02ddfb82d3f4294b2f822359e0f4247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } } }
I20260926 14:56:00.758165 8938 tablet_replica.cc:442] T 5b8e3b2f47c94477801e07c4ccb2892c P ea025850c67845d2b60aade34eafdb63: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:00.758167 8940 tablet_replica.cc:442] T 5b8e3b2f47c94477801e07c4ccb2892c P bbd48da2238e479ba63c3c42fcc143ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:00.758816 8944 tablet_replica.cc:439] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f02ddfb82d3f4294b2f822359e0f4247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } } }
I20260926 14:56:00.758888 8944 tablet_replica.cc:442] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247: This TxnStatusTablet replica's current role is: LEADER
I20260926 14:56:00.759050 8940 tablet_replica.cc:439] T 5b8e3b2f47c94477801e07c4ccb2892c P bbd48da2238e479ba63c3c42fcc143ff: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f02ddfb82d3f4294b2f822359e0f4247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } } }
I20260926 14:56:00.759130 8940 tablet_replica.cc:442] T 5b8e3b2f47c94477801e07c4ccb2892c P bbd48da2238e479ba63c3c42fcc143ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:00.759260 8952 tablet_replica.cc:439] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f02ddfb82d3f4294b2f822359e0f4247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } } }
I20260926 14:56:00.759322 8952 tablet_replica.cc:442] T 5b8e3b2f47c94477801e07c4ccb2892c P f02ddfb82d3f4294b2f822359e0f4247: This TxnStatusTablet replica's current role is: LEADER
I20260926 14:56:00.759373 8938 tablet_replica.cc:439] T 5b8e3b2f47c94477801e07c4ccb2892c P ea025850c67845d2b60aade34eafdb63: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f02ddfb82d3f4294b2f822359e0f4247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02ddfb82d3f4294b2f822359e0f4247" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33479 } } peers { permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 } } peers { permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 } } }
I20260926 14:56:00.759611 8938 tablet_replica.cc:442] T 5b8e3b2f47c94477801e07c4ccb2892c P ea025850c67845d2b60aade34eafdb63: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:00.764869 8981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.762200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:33479 (local address 127.7.37.193:58331)
0926 14:56:00.762354 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:00.762362 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:00.762407 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:00.762507 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:00.762507 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:00.762508 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 14:56:00.762540 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.762541 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.763353 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.763354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.764007 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:00.764009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.764441 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.764442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:00.764486 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.764498 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 14:56:00.764705 (+ 207us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 14:56:00.764707 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:56:00.764758 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 14:56:00.764800 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":1,"thread_start_us":66,"threads_started":1}
I20260926 14:56:00.764873 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.762258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:58331 (local address 127.7.37.193:33479)
0926 14:56:00.762297 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:00.762298 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:00.762411 (+ 113us) server_negotiation.cc:407] Connection header received
0926 14:56:00.762449 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:00.762450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:00.762459 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:00.762505 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:00.762575 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.763290 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.764071 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.764356 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.764421 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.764592 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:00.764632 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:00.764679 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:00.764770 (+ 91us) server_negotiation.cc:299] Negotiation successful
0926 14:56:00.764795 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":22}
I20260926 14:56:00.766402 8728 raft_consensus.cc:1261] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff [term 1 FOLLOWER]: Refusing update from remote peer f02ddfb82d3f4294b2f822359e0f4247: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:00.766510 8868 raft_consensus.cc:1261] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63 [term 1 FOLLOWER]: Refusing update from remote peer f02ddfb82d3f4294b2f822359e0f4247: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:00.766587 8942 consensus_queue.cc:1035] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbd48da2238e479ba63c3c42fcc143ff" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:00.766674 8952 consensus_queue.cc:1035] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea025850c67845d2b60aade34eafdb63" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:00.776223 8430 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:00.776297 8430 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:00.776822 8430 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04a7136da95d44f29413545a9cad927f: Sending DeleteTablet for 3 replicas of tablet 0b6c68642d6a4556961691037ac8d8dd
I20260926 14:56:00.777040 8708 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b6c68642d6a4556961691037ac8d8dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:00 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:36386
I20260926 14:56:00.777084 8568 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b6c68642d6a4556961691037ac8d8dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:00 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:38772
I20260926 14:56:00.777088 8848 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b6c68642d6a4556961691037ac8d8dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:00 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:55744
I20260926 14:56:00.777133 8958 tablet_replica.cc:330] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff: stopping tablet replica
I20260926 14:56:00.777144 8957 tablet_replica.cc:330] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247: stopping tablet replica
I20260926 14:56:00.777172 8958 raft_consensus.cc:2229] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:00.777184 8957 raft_consensus.cc:2229] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:56:00.777199 8958 raft_consensus.cc:2258] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:00.777225 8957 raft_consensus.cc:2258] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:00.777289 8959 tablet_replica.cc:330] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63: stopping tablet replica
I20260926 14:56:00.777335 8959 raft_consensus.cc:2229] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:00.777362 8959 raft_consensus.cc:2258] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:00.777520 8957 ts_tablet_manager.cc:1905] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:00.778132 8959 ts_tablet_manager.cc:1905] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:00.779012 8957 ts_tablet_manager.cc:1918] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 14:56:00.779078 8957 log.cc:1196] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-0/wal/wals/0b6c68642d6a4556961691037ac8d8dd
I20260926 14:56:00.779312 8957 ts_tablet_manager.cc:1939] T 0b6c68642d6a4556961691037ac8d8dd P f02ddfb82d3f4294b2f822359e0f4247: Deleting consensus metadata
I20260926 14:56:00.779492 8959 ts_tablet_manager.cc:1918] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 14:56:00.779546 8959 log.cc:1196] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/0b6c68642d6a4556961691037ac8d8dd
I20260926 14:56:00.779767 8959 ts_tablet_manager.cc:1939] T 0b6c68642d6a4556961691037ac8d8dd P ea025850c67845d2b60aade34eafdb63: Deleting consensus metadata
I20260926 14:56:00.780055 8418 catalog_manager.cc:4841] TS ea025850c67845d2b60aade34eafdb63 (127.7.37.195:33283): tablet 0b6c68642d6a4556961691037ac8d8dd (table test-table [id=4b6eb64f810e4179ac7f189386f2fb77]) successfully deleted
I20260926 14:56:00.780377 8415 catalog_manager.cc:4841] TS f02ddfb82d3f4294b2f822359e0f4247 (127.7.37.193:33479): tablet 0b6c68642d6a4556961691037ac8d8dd (table test-table [id=4b6eb64f810e4179ac7f189386f2fb77]) successfully deleted
Sep 26 14:56:00 dist-test-slave-f9xb krb5kdc[8386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434560, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.37.193@KRBTEST.COM
I20260926 14:56:00.786123 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.778113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36400 (local address 127.7.37.193:33479)
0926 14:56:00.778148 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:00.778149 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:00.778446 (+ 297us) server_negotiation.cc:407] Connection header received
0926 14:56:00.778543 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:00.778543 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:00.778553 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:00.778582 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:00.779502 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.780114 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.781051 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.781251 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.781303 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.783782 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:00.783791 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:00.783791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:00.783798 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:00.785151 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:00.785435 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:00.785438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:00.785439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:00.785496 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:00.785656 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:00.785657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:00.785657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:00.785780 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:00.785899 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:00.785959 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 14:56:00.786029 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 14:56:00.786343 8958 ts_tablet_manager.cc:1905] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 14:56:00.786377 8611 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36400
I20260926 14:56:00.787637 8958 ts_tablet_manager.cc:1918] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 14:56:00.787684 8958 log.cc:1196] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/0b6c68642d6a4556961691037ac8d8dd
I20260926 14:56:00.787871 8958 ts_tablet_manager.cc:1939] T 0b6c68642d6a4556961691037ac8d8dd P bbd48da2238e479ba63c3c42fcc143ff: Deleting consensus metadata
I20260926 14:56:00.788213 8416 catalog_manager.cc:4841] TS bbd48da2238e479ba63c3c42fcc143ff (127.7.37.194:37893): tablet 0b6c68642d6a4556961691037ac8d8dd (table test-table [id=4b6eb64f810e4179ac7f189386f2fb77]) successfully deleted
I20260926 14:56:00.797665 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:00.787223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53134 (local address 127.7.37.254:46637)
0926 14:56:00.787261 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:00.787263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:00.792117 (+ 4854us) server_negotiation.cc:407] Connection header received
0926 14:56:00.792170 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:00.792171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:00.792181 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:00.792224 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:00.792638 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.793146 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.794105 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:00.794274 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:00.794311 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:00.795511 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:00.795512 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:00.795512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:00.795518 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:00.796757 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:00.797084 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:00.797084 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:00.797084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:00.797126 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:00.797286 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:00.797286 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:00.797286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:00.797375 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:00.797435 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:00.797541 (+ 106us) server_negotiation.cc:299] Negotiation successful
0926 14:56:00.797614 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260926 14:56:00.797938 8430 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53134
I20260926 14:56:00.798439 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 8503
I20260926 14:56:00.803182 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 8643
I20260926 14:56:00.806772 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 8783
I20260926 14:56:00.810186 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 8399
2026-09-26T14:56:00Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2220 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:56:00 dist-test-slave-f9xb krb5kdc[8991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:56:00 dist-test-slave-f9xb krb5kdc[8991](info): set up 2 sockets
Sep 26 14:56:00 dist-test-slave-f9xb krb5kdc[8991](info): commencing operation
krb5kdc: starting...
W20260926 14:56:01.747079 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.918s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:56:01 dist-test-slave-f9xb krb5kdc[8991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:56:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:56:01.847978 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:42985
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:35979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:42985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:01.921933 9004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:01.922107 9004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:01.922135 9004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:01.923211 9004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:01.923249 9004 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 14:56:01.923266 9004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:01.923282 9004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:01.923298 9004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:01.923321 9004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:56:01.924742 9004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:35979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:42985
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:42985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:01.924955 9004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:01.925141 9004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:01.929299 9009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:01.929406 9010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:01.929458 9012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:01.929672 9004 server_base.cc:1034] running on GCE node
I20260926 14:56:01.929876 9004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:01.930111 9004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:01.931258 9004 hybrid_clock.cc:648] HybridClock initialized: now 1790434561931243 us; error 26 us; skew 500 ppm
Sep 26 14:56:01 dist-test-slave-f9xb krb5kdc[8991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434561, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:01.933419 9004 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:56:01.933794 9004 webserver.cc:460] Webserver started at http://127.7.37.254:36913/ using document root <none> and password file <none>
I20260926 14:56:01.933969 9004 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:01.934012 9004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:01.934125 9004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:01.935027 9004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "e83be67a509c459ca1e017fc811a2f3a"
format_stamp: "Formatted at 2026-09-26 14:56:01 on dist-test-slave-f9xb"
server_key: "01b9c4d0ce7929c40d581f3855221ec4"
server_key_iv: "19a0f44cf9f7f32cbed6a828a976d9aa"
server_key_version: "encryptionkey@0"
I20260926 14:56:01.935322 9004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "e83be67a509c459ca1e017fc811a2f3a"
format_stamp: "Formatted at 2026-09-26 14:56:01 on dist-test-slave-f9xb"
server_key: "01b9c4d0ce7929c40d581f3855221ec4"
server_key_iv: "19a0f44cf9f7f32cbed6a828a976d9aa"
server_key_version: "encryptionkey@0"
I20260926 14:56:01.936523 9004 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 14:56:01.937242 9019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:01.937415 9004 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:01.937492 9004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "e83be67a509c459ca1e017fc811a2f3a"
format_stamp: "Formatted at 2026-09-26 14:56:01 on dist-test-slave-f9xb"
server_key: "01b9c4d0ce7929c40d581f3855221ec4"
server_key_iv: "19a0f44cf9f7f32cbed6a828a976d9aa"
server_key_version: "encryptionkey@0"
I20260926 14:56:01.937556 9004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:01.955565 9004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:01.957041 9004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:01.957156 9004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:01.961031 9004 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:42985
I20260926 14:56:01.961112 9081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:42985 every 8 connection(s)
I20260926 14:56:01.961345 9004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:56:01.961920 9082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:01.963892 9082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a: Bootstrap starting.
I20260926 14:56:01.964380 9082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:01.964628 9082 log.cc:824] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:01.965175 9082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a: No bootstrap required, opened a new log
I20260926 14:56:01.967077 9082 raft_consensus.cc:348] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83be67a509c459ca1e017fc811a2f3a" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42985 } }
I20260926 14:56:01.967195 9082 raft_consensus.cc:374] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:01.967216 9082 raft_consensus.cc:729] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e83be67a509c459ca1e017fc811a2f3a, State: Initialized, Role: FOLLOWER
I20260926 14:56:01.967342 9082 consensus_queue.cc:260] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83be67a509c459ca1e017fc811a2f3a" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42985 } }
I20260926 14:56:01.967401 9082 raft_consensus.cc:388] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:56:01.967423 9082 raft_consensus.cc:482] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:56:01.967466 9082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:01.967969 9082 raft_consensus.cc:504] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83be67a509c459ca1e017fc811a2f3a" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42985 } }
I20260926 14:56:01.968051 9082 leader_election.cc:302] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e83be67a509c459ca1e017fc811a2f3a; no voters:
I20260926 14:56:01.968195 9082 leader_election.cc:288] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:56:01.968230 9085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:01.968362 9085 raft_consensus.cc:686] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a [term 1 LEADER]: Becoming Leader. State: Replica: e83be67a509c459ca1e017fc811a2f3a, State: Running, Role: LEADER
I20260926 14:56:01.968415 9082 sys_catalog.cc:564] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:01.968467 9085 consensus_queue.cc:237] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83be67a509c459ca1e017fc811a2f3a" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42985 } }
I20260926 14:56:01.968909 9087 sys_catalog.cc:455] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e83be67a509c459ca1e017fc811a2f3a. Latest consensus state: current_term: 1 leader_uuid: "e83be67a509c459ca1e017fc811a2f3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83be67a509c459ca1e017fc811a2f3a" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42985 } } }
I20260926 14:56:01.968924 9086 sys_catalog.cc:455] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e83be67a509c459ca1e017fc811a2f3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83be67a509c459ca1e017fc811a2f3a" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42985 } } }
I20260926 14:56:01.969007 9087 sys_catalog.cc:458] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:01.969060 9086 sys_catalog.cc:458] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:01.969216 9094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:56:01.969741 9094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:56:01.971124 9094 catalog_manager.cc:1347] Generated new cluster ID: bf3fdf1739fb48fb8f4e2e95780eaf3d
I20260926 14:56:01.971179 9094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:56:01.971390 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 9004
I20260926 14:56:01.971489 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:56:01.971683 7319 external_mini_cluster.cc:1428] Setting key 2b93eefae45303ee277235127f0834ee
Sep 26 14:56:01 dist-test-slave-f9xb krb5kdc[8991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:01.978964 9094 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:56:01.979501 9094 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:01.982456 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:01.972122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45738 (local address 127.7.37.254:42985)
0926 14:56:01.972271 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:01.972280 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:01.972357 (+ 77us) server_negotiation.cc:407] Connection header received
0926 14:56:01.972506 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:01.972512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:01.972564 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:01.972703 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:01.973394 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:01.974762 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:01.975783 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:01.976381 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:01.976481 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:01.979126 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:01.979130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:01.979130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:01.979141 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:01.980999 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:01.981384 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:01.981387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:01.981387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:01.981485 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:01.981655 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:01.981656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:01.981656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:01.981892 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:01.982016 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:01.982118 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 14:56:01.982251 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
I20260926 14:56:01.985981 9094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a: Generated new TSK 0
I20260926 14:56:01.986125 9094 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
I20260926 14:56:02.010834 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42985
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:35979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:02.085664 9108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:02.085840 9108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:02.085872 9108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:02.086956 9108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:02.086993 9108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:02.087019 9108 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 14:56:02.087034 9108 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 14:56:02.087057 9108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:56:02.088434 9108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:35979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.37.254:42985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:02.088644 9108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:02.088824 9108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:02.093189 9116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:02.093258 9113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:02.093245 9114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:02.093396 9108 server_base.cc:1034] running on GCE node
I20260926 14:56:02.093580 9108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:02.093772 9108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:02.094898 9108 hybrid_clock.cc:648] HybridClock initialized: now 1790434562094888 us; error 27 us; skew 500 ppm
Sep 26 14:56:02 dist-test-slave-f9xb krb5kdc[8991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434562, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:02.097055 9108 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:56:02.097434 9108 webserver.cc:460] Webserver started at http://127.7.37.193:46567/ using document root <none> and password file <none>
I20260926 14:56:02.097600 9108 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:02.097644 9108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:02.097728 9108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:02.098589 9108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "ce649b409ece4d3483a8c8920e127e61"
format_stamp: "Formatted at 2026-09-26 14:56:02 on dist-test-slave-f9xb"
server_key: "b39314e0e19c03c2b8b290dfaa961be3"
server_key_iv: "c591b6f5d984120bceaefb0a4925d2c6"
server_key_version: "encryptionkey@0"
I20260926 14:56:02.098845 9108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "ce649b409ece4d3483a8c8920e127e61"
format_stamp: "Formatted at 2026-09-26 14:56:02 on dist-test-slave-f9xb"
server_key: "b39314e0e19c03c2b8b290dfaa961be3"
server_key_iv: "c591b6f5d984120bceaefb0a4925d2c6"
server_key_version: "encryptionkey@0"
I20260926 14:56:02.099907 9108 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:02.100493 9123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:02.100634 9108 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:02.100703 9108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "ce649b409ece4d3483a8c8920e127e61"
format_stamp: "Formatted at 2026-09-26 14:56:02 on dist-test-slave-f9xb"
server_key: "b39314e0e19c03c2b8b290dfaa961be3"
server_key_iv: "c591b6f5d984120bceaefb0a4925d2c6"
server_key_version: "encryptionkey@0"
I20260926 14:56:02.100761 9108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:02.113994 9108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:02.115460 9108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:02.115576 9108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:02.116036 9108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:02.116076 9108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:02.116096 9108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:02.116107 9108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 14:56:02 dist-test-slave-f9xb krb5kdc[8991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434562, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:02.122352 9108 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:37455
I20260926 14:56:02.122426 9238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:37455 every 8 connection(s)
I20260926 14:56:02.122668 9108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:56:02.124576 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.116352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:44211 (local address 127.7.37.254:42985)
0926 14:56:02.116394 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:02.116397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:02.116614 (+ 217us) server_negotiation.cc:407] Connection header received
0926 14:56:02.117000 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:02.117002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:02.117013 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:02.117077 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:02.117579 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.118299 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.119276 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.119566 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.119634 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.122165 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:02.122166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:02.122167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:02.122173 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:02.123542 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:02.123916 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:02.123916 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:02.123917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:02.123962 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:02.124113 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:02.124114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:02.124114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:02.124200 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:02.124241 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:02.124403 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 14:56:02.124470 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:56:02.124585 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 9108
I20260926 14:56:02.124627 9132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.116335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42985 (local address 127.7.37.193:44211)
0926 14:56:02.116500 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:02.116535 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:02.116921 (+ 386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:02.117120 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:02.117123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:02.117325 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:02.117512 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.117516 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.118388 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.118389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.119211 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.119214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.119638 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.119639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.119790 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.120227 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:02.120235 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:02.122083 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:02.123613 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:02.123616 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:02.123618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:02.123860 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:02.124000 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:02.124000 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:02.124000 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:02.124081 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:02.124233 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:02.124235 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:02.124325 (+ 90us) client_negotiation.cc:769] Sending connection context
0926 14:56:02.124391 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 14:56:02.124493 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260926 14:56:02.124667 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:56:02.124818 7319 external_mini_cluster.cc:1428] Setting key 99b93ecacbb629e89298baf580bc31c9
I20260926 14:56:02.125025 9239 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42985
I20260926 14:56:02.125087 9239 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:02.125202 9239 heartbeater.cc:511] Master 127.7.37.254:42985 requested a full tablet report, sending...
I20260926 14:56:02.125644 9036 ts_manager.cc:194] Registered new tserver with Master: ce649b409ece4d3483a8c8920e127e61 (127.7.37.193:37455)
I20260926 14:56:02.126482 9036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:44211
I20260926 14:56:02.131322 9245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.125872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42985 (local address 127.7.37.193:49751)
0926 14:56:02.126187 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:02.126197 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:02.126260 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:02.126381 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:02.126382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:02.126535 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:02.126596 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.126598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.127399 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.127400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.127948 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.127950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.128254 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.128255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.128321 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.128631 (+ 310us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:02.128636 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:02.128939 (+ 303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:02.130533 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:02.130533 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:02.130534 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:02.130679 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:02.130852 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:02.130853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:02.130853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:02.130895 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:02.131117 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:02.131117 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:02.131148 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 14:56:02.131191 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 14:56:02.131266 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":60,"threads_started":1}
I20260926 14:56:02.131376 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.125922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:49751 (local address 127.7.37.254:42985)
0926 14:56:02.126135 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:02.126137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:02.126239 (+ 102us) server_negotiation.cc:407] Connection header received
0926 14:56:02.126327 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:02.126328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:02.126337 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:02.126379 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:02.126651 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.127314 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.128016 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.128203 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.128251 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.129025 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:02.129026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:02.129027 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:02.129032 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:02.130421 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:02.130741 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:02.130741 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:02.130742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:02.130794 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:02.130940 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:02.130941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:02.130941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:02.131054 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:02.131118 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:02.131230 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 14:56:02.131294 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197}
WARNING: no policy specified for kudu/127.7.37.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
I20260926 14:56:02.147931 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42985
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:35979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:02.223706 9248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:02.223899 9248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:02.223920 9248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:02.225072 9248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:02.225111 9248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:02.225131 9248 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 14:56:02.225154 9248 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 14:56:02.225185 9248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:56:02.226608 9248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:35979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.37.254:42985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:02.226804 9248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:02.226994 9248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:02.231329 9256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:02.231338 9253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:02.231356 9248 server_base.cc:1034] running on GCE node
W20260926 14:56:02.231328 9254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:02.231693 9248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:02.231920 9248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:02.233057 9248 hybrid_clock.cc:648] HybridClock initialized: now 1790434562233044 us; error 28 us; skew 500 ppm
Sep 26 14:56:02 dist-test-slave-f9xb krb5kdc[8991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434562, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:02.235338 9248 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:56:02.235697 9248 webserver.cc:460] Webserver started at http://127.7.37.194:45475/ using document root <none> and password file <none>
I20260926 14:56:02.235893 9248 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:02.235934 9248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:02.236024 9248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:02.236902 9248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "b4af784ae2f34378ab2dafec3ae54481"
format_stamp: "Formatted at 2026-09-26 14:56:02 on dist-test-slave-f9xb"
server_key: "a026dcc00c77dc53fa9971440965d7af"
server_key_iv: "a47d1621d0b020f15fd518e1c4100757"
server_key_version: "encryptionkey@0"
I20260926 14:56:02.237177 9248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "b4af784ae2f34378ab2dafec3ae54481"
format_stamp: "Formatted at 2026-09-26 14:56:02 on dist-test-slave-f9xb"
server_key: "a026dcc00c77dc53fa9971440965d7af"
server_key_iv: "a47d1621d0b020f15fd518e1c4100757"
server_key_version: "encryptionkey@0"
I20260926 14:56:02.238219 9248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:02.238776 9263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:02.238909 9248 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:02.238973 9248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "b4af784ae2f34378ab2dafec3ae54481"
format_stamp: "Formatted at 2026-09-26 14:56:02 on dist-test-slave-f9xb"
server_key: "a026dcc00c77dc53fa9971440965d7af"
server_key_iv: "a47d1621d0b020f15fd518e1c4100757"
server_key_version: "encryptionkey@0"
I20260926 14:56:02.239037 9248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:02.250195 9248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:02.252102 9248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:02.252209 9248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:02.252620 9248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:02.252658 9248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:02.252687 9248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:02.252710 9248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 14:56:02 dist-test-slave-f9xb krb5kdc[8991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434562, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:02.258508 9248 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:44771
I20260926 14:56:02.258567 9378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:44771 every 8 connection(s)
I20260926 14:56:02.258850 9248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:56:02.261197 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.253007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:43909 (local address 127.7.37.254:42985)
0926 14:56:02.253072 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:02.253075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:02.253253 (+ 178us) server_negotiation.cc:407] Connection header received
0926 14:56:02.253649 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:02.253650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:02.253662 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:02.253737 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:02.254400 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.255151 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.256342 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.256636 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.256701 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.259013 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:02.259015 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:02.259015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:02.259022 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:02.260256 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:02.260569 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:02.260570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:02.260570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:02.260613 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:02.260776 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:02.260777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:02.260777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:02.260861 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:02.260903 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:02.261088 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 14:56:02.261140 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 14:56:02.261327 9272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.252988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42985 (local address 127.7.37.194:43909)
0926 14:56:02.253130 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:02.253165 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:02.253564 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:02.253780 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:02.253783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:02.254057 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:02.254305 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.254310 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.255252 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.255253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.256265 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.256267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.256717 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.256718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.256849 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.257270 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:02.257278 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:02.258924 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:02.260316 (+ 1392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:02.260321 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:02.260322 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:02.260513 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:02.260655 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:02.260655 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:02.260655 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:02.260736 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:02.260902 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:02.260904 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:02.260999 (+ 95us) client_negotiation.cc:769] Sending connection context
0926 14:56:02.261069 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 14:56:02.261169 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260926 14:56:02.261472 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 9248
I20260926 14:56:02.261556 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:56:02.261721 7319 external_mini_cluster.cc:1428] Setting key 8a0cf6ea265df679d0b35b6e234ffd85
I20260926 14:56:02.261795 9379 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42985
I20260926 14:56:02.261862 9379 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:02.262008 9379 heartbeater.cc:511] Master 127.7.37.254:42985 requested a full tablet report, sending...
I20260926 14:56:02.262336 9036 ts_manager.cc:194] Registered new tserver with Master: b4af784ae2f34378ab2dafec3ae54481 (127.7.37.194:44771)
I20260926 14:56:02.263034 9036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:43909
I20260926 14:56:02.268319 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.262752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:35157 (local address 127.7.37.254:42985)
0926 14:56:02.263190 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:02.263192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:02.263199 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:02.263215 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:02.263215 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:02.263223 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:02.263267 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:02.263895 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.264394 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.265187 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.265362 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.265409 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.266130 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:02.266131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:02.266131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:02.266137 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:02.267345 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:02.267608 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:02.267609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:02.267609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:02.267656 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:02.267867 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:02.267868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:02.267868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:02.267995 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:02.268049 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:02.268182 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 14:56:02.268239 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424}
I20260926 14:56:02.268322 9385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.262660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42985 (local address 127.7.37.194:35157)
0926 14:56:02.262788 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:02.262797 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:02.262840 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:02.263607 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:02.263608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:02.263783 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:02.263834 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.263835 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.264472 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.264473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.265102 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.265104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.265413 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.265414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.265482 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.265817 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:02.265821 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:02.266056 (+ 235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:02.267409 (+ 1353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:02.267410 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:02.267410 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:02.267555 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:02.267762 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:02.267762 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:02.267763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:02.267817 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:02.268047 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:02.268048 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:02.268089 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 14:56:02.268148 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 14:56:02.268235 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for kudu/127.7.37.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
I20260926 14:56:02.284138 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42985
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:35979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:02.371237 9388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:02.371399 9388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:02.371420 9388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:02.372530 9388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:02.372568 9388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:02.372601 9388 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 14:56:02.372612 9388 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 14:56:02.372627 9388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:56:02.373945 9388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:35979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.37.254:42985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:02.374104 9388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:02.374259 9388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:02.378646 9396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:02.378647 9394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:02.378772 9388 server_base.cc:1034] running on GCE node
W20260926 14:56:02.378726 9393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:02.378961 9388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:02.379170 9388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:02.380304 9388 hybrid_clock.cc:648] HybridClock initialized: now 1790434562380290 us; error 26 us; skew 500 ppm
Sep 26 14:56:02 dist-test-slave-f9xb krb5kdc[8991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434562, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:02.382401 9388 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:56:02.382769 9388 webserver.cc:460] Webserver started at http://127.7.37.195:45035/ using document root <none> and password file <none>
I20260926 14:56:02.382952 9388 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:02.382995 9388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:02.383113 9388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:02.384012 9388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "b060492dd71a43b987c33636317ecd17"
format_stamp: "Formatted at 2026-09-26 14:56:02 on dist-test-slave-f9xb"
server_key: "d29086273843e6843864a6e411c410bb"
server_key_iv: "b5b54f80fe8a766397cd373fce4c4540"
server_key_version: "encryptionkey@0"
I20260926 14:56:02.384302 9388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "b060492dd71a43b987c33636317ecd17"
format_stamp: "Formatted at 2026-09-26 14:56:02 on dist-test-slave-f9xb"
server_key: "d29086273843e6843864a6e411c410bb"
server_key_iv: "b5b54f80fe8a766397cd373fce4c4540"
server_key_version: "encryptionkey@0"
I20260926 14:56:02.385376 9388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:02.385967 9403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:02.386111 9388 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:02.386181 9388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "b060492dd71a43b987c33636317ecd17"
format_stamp: "Formatted at 2026-09-26 14:56:02 on dist-test-slave-f9xb"
server_key: "d29086273843e6843864a6e411c410bb"
server_key_iv: "b5b54f80fe8a766397cd373fce4c4540"
server_key_version: "encryptionkey@0"
I20260926 14:56:02.386235 9388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:02.421407 9388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:02.422931 9388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:02.423044 9388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:02.423441 9388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:02.423480 9388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:02.423506 9388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:02.423523 9388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 14:56:02 dist-test-slave-f9xb krb5kdc[8991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434562, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:02.429911 9388 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:42517
I20260926 14:56:02.430226 9518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:42517 every 8 connection(s)
I20260926 14:56:02.430239 9388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:56:02.431875 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.423818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:39211 (local address 127.7.37.254:42985)
0926 14:56:02.423886 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:02.423888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:02.424084 (+ 196us) server_negotiation.cc:407] Connection header received
0926 14:56:02.424633 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:02.424634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:02.424644 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:02.424682 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:02.425398 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.425897 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.426953 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.427143 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.427195 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.429493 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:02.429495 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:02.429495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:02.429501 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:02.430884 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:02.431207 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:02.431208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:02.431209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:02.431252 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:02.431401 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:02.431401 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:02.431402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:02.431487 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:02.431527 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:02.431724 (+ 197us) server_negotiation.cc:299] Negotiation successful
0926 14:56:02.431815 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 14:56:02.432013 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.423811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42985 (local address 127.7.37.195:39211)
0926 14:56:02.423959 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:02.424000 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:02.424532 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:02.424749 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:02.424753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:02.425064 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:02.425321 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.425326 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.425973 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.425974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.426872 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.426874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.427205 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.427205 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.427327 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.427790 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:02.427799 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:02.429415 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:02.430945 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:02.430948 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:02.430949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:02.431145 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:02.431286 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:02.431287 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:02.431287 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:02.431364 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:02.431529 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:02.431531 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:02.431634 (+ 103us) client_negotiation.cc:769] Sending connection context
0926 14:56:02.431706 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 14:56:02.431853 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
I20260926 14:56:02.432396 9519 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42985
I20260926 14:56:02.432461 9519 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:02.432613 9519 heartbeater.cc:511] Master 127.7.37.254:42985 requested a full tablet report, sending...
I20260926 14:56:02.432912 9036 ts_manager.cc:194] Registered new tserver with Master: b060492dd71a43b987c33636317ecd17 (127.7.37.195:42517)
I20260926 14:56:02.433385 9036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:39211
I20260926 14:56:02.437551 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.433272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:59161 (local address 127.7.37.254:42985)
0926 14:56:02.433497 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:02.433498 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:02.433503 (+ 5us) server_negotiation.cc:407] Connection header received
0926 14:56:02.433517 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:02.433518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:02.433526 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:02.433572 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:02.433849 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.434272 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.434900 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.435067 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.435125 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.435838 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:02.435839 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:02.435839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:02.435844 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:02.436802 (+ 958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:02.437053 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:02.437053 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:02.437054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:02.437102 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:02.437229 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:02.437230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:02.437230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:02.437322 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:02.437367 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:02.437453 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 14:56:02.437500 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212}
I20260926 14:56:02.437561 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.433210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42985 (local address 127.7.37.195:59161)
0926 14:56:02.433322 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:02.433331 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:02.433378 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:02.433579 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:02.433580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:02.433729 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:02.433783 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.433785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.434343 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.434344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.434838 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.434839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.435122 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.435123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.435172 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.435504 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:02.435508 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:02.435776 (+ 268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:02.436859 (+ 1083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:02.436860 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:02.436861 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:02.437004 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:02.437142 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:02.437142 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:02.437142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:02.437184 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:02.437365 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:02.437365 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:02.437395 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 14:56:02.437437 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 14:56:02.437498 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":73,"threads_started":1}
I20260926 14:56:02.437855 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 9388
I20260926 14:56:02.437945 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:56:02.438123 7319 external_mini_cluster.cc:1428] Setting key f8baac0d1269ccae124e8cce3bee3a91
I20260926 14:56:02.438895 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 14:56:02 dist-test-slave-f9xb krb5kdc[8991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 14:56:02 dist-test-slave-f9xb krb5kdc[8991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:02.458012 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.452439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45752 (local address 127.7.37.254:42985)
0926 14:56:02.452477 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:02.452479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:02.452487 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:02.452532 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:02.452533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:02.452539 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:02.452582 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:02.453029 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.453475 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.454132 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.454298 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.454340 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.456179 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:02.456180 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:02.456180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:02.456185 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:02.457287 (+ 1102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:02.457528 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:02.457528 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:02.457529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:02.457567 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:02.457688 (+ 121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:02.457689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:02.457689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:02.457765 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:02.457827 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:02.457896 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 14:56:02.457957 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 14:56:02.459280 9036 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:56:02.460775 9036 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:56:02.466311 9533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.462671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:42517 (local address 127.0.0.1:53488)
0926 14:56:02.463060 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:02.463070 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:02.463130 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:02.463276 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:02.463277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:02.463279 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:02.463326 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.463328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.464266 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.464267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.465199 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.465201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.466046 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.466047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.466119 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.466132 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 14:56:02.466196 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 14:56:02.466248 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":46,"threads_started":1}
I20260926 14:56:02.466327 9537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.462874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53488 (local address 127.7.37.195:42517)
0926 14:56:02.463018 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:02.463021 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:02.463127 (+ 106us) server_negotiation.cc:407] Connection header received
0926 14:56:02.463189 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:02.463191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:02.463205 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:02.463267 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:02.463396 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.464200 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.465301 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.465993 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.466048 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.466112 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:02.466210 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 14:56:02.466244 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":66,"threads_started":1}
I20260926 14:56:02.466393 9534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.462649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:44771 (local address 127.0.0.1:52466)
0926 14:56:02.462812 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:02.462936 (+ 124us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:02.463002 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:02.463238 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:02.463240 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:02.463242 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:02.463294 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.463296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.464523 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.464524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.465467 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.465469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.466181 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.466182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.466248 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.466251 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:02.466294 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 14:56:02.466327 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":74,"threads_started":1}
I20260926 14:56:02.466650 9535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.462724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52466 (local address 127.7.37.194:44771)
0926 14:56:02.462862 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:02.462867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:02.462990 (+ 123us) server_negotiation.cc:407] Connection header received
0926 14:56:02.463077 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:02.463080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:02.463103 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:02.463168 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:02.463349 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.464435 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.465529 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.466024 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.466096 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.466148 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:02.466527 (+ 379us) server_negotiation.cc:299] Negotiation successful
0926 14:56:02.466565 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260926 14:56:02.467123 9453 tablet_service.cc:1467] Processing CreateTablet for tablet a7459c81afb845a4a2ed01084fc103a1 (DEFAULT_TABLE table=test-table [id=0a3f696841c84041bea0bbfa22a3e518]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:02.467170 9313 tablet_service.cc:1467] Processing CreateTablet for tablet a7459c81afb845a4a2ed01084fc103a1 (DEFAULT_TABLE table=test-table [id=0a3f696841c84041bea0bbfa22a3e518]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:02.467410 9453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7459c81afb845a4a2ed01084fc103a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:02.467422 9313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7459c81afb845a4a2ed01084fc103a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:02.469880 9539 tablet_bootstrap.cc:492] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17: Bootstrap starting.
I20260926 14:56:02.470515 9540 tablet_bootstrap.cc:492] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481: Bootstrap starting.
I20260926 14:56:02.470541 9536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.462846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:37455 (local address 127.0.0.1:59074)
0926 14:56:02.463226 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:02.463234 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:02.463325 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:02.463616 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:02.463617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:02.463619 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:02.463657 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.463658 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.464867 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.464868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.465901 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.465903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.470329 (+ 4426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.470330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.470393 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.470397 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:56:02.470442 (+ 45us) client_negotiation.cc:240] Negotiation successful
0926 14:56:02.470479 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":69,"threads_started":1}
I20260926 14:56:02.470844 9538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.462941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59074 (local address 127.7.37.193:37455)
0926 14:56:02.463287 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:02.463293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:02.463303 (+ 10us) server_negotiation.cc:407] Connection header received
0926 14:56:02.463459 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:02.463461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:02.463475 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:02.463537 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:02.463728 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.464792 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.465982 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.470256 (+ 4274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.470317 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.470366 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:02.470751 (+ 385us) server_negotiation.cc:299] Negotiation successful
0926 14:56:02.470780 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":56,"threads_started":1}
I20260926 14:56:02.470985 9539 tablet_bootstrap.cc:654] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:02.471256 9540 tablet_bootstrap.cc:654] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:02.471313 9539 log.cc:824] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:02.471343 9173 tablet_service.cc:1467] Processing CreateTablet for tablet a7459c81afb845a4a2ed01084fc103a1 (DEFAULT_TABLE table=test-table [id=0a3f696841c84041bea0bbfa22a3e518]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:02.471555 9540 log.cc:824] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:02.471622 9173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7459c81afb845a4a2ed01084fc103a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:02.472021 9539 tablet_bootstrap.cc:492] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17: No bootstrap required, opened a new log
I20260926 14:56:02.472080 9539 ts_tablet_manager.cc:1397] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:56:02.472141 9540 tablet_bootstrap.cc:492] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481: No bootstrap required, opened a new log
I20260926 14:56:02.472182 9540 ts_tablet_manager.cc:1397] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:56:02.473356 9540 raft_consensus.cc:348] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.473461 9540 raft_consensus.cc:374] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:02.473479 9540 raft_consensus.cc:729] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4af784ae2f34378ab2dafec3ae54481, State: Initialized, Role: FOLLOWER
I20260926 14:56:02.473563 9540 consensus_queue.cc:260] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.473785 9540 ts_tablet_manager.cc:1428] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:56:02.473804 9379 heartbeater.cc:503] Master 127.7.37.254:42985 was elected leader, sending a full tablet report...
I20260926 14:56:02.473865 9539 raft_consensus.cc:348] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.473999 9539 raft_consensus.cc:374] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:02.474027 9539 raft_consensus.cc:729] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b060492dd71a43b987c33636317ecd17, State: Initialized, Role: FOLLOWER
I20260926 14:56:02.474131 9539 consensus_queue.cc:260] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.474350 9539 ts_tablet_manager.cc:1428] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 14:56:02.474381 9519 heartbeater.cc:503] Master 127.7.37.254:42985 was elected leader, sending a full tablet report...
I20260926 14:56:02.474941 9545 tablet_bootstrap.cc:492] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61: Bootstrap starting.
I20260926 14:56:02.475386 9545 tablet_bootstrap.cc:654] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:02.475630 9545 log.cc:824] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:02.476187 9545 tablet_bootstrap.cc:492] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61: No bootstrap required, opened a new log
I20260926 14:56:02.476233 9545 ts_tablet_manager.cc:1397] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 14:56:02.477483 9545 raft_consensus.cc:348] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.477599 9545 raft_consensus.cc:374] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:02.477619 9545 raft_consensus.cc:729] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce649b409ece4d3483a8c8920e127e61, State: Initialized, Role: FOLLOWER
I20260926 14:56:02.477698 9545 consensus_queue.cc:260] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.477893 9545 ts_tablet_manager.cc:1428] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 14:56:02.477953 9239 heartbeater.cc:503] Master 127.7.37.254:42985 was elected leader, sending a full tablet report...
W20260926 14:56:02.509320 9380 tablet.cc:2369] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:56:02.573119 9547 raft_consensus.cc:482] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:56:02.573220 9547 raft_consensus.cc:504] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.573534 9547 leader_election.cc:288] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4af784ae2f34378ab2dafec3ae54481 (127.7.37.194:44771), b060492dd71a43b987c33636317ecd17 (127.7.37.195:42517)
I20260926 14:56:02.577117 9537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.573805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:52231 (local address 127.7.37.195:42517)
0926 14:56:02.573859 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:02.573861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:02.573895 (+ 34us) server_negotiation.cc:407] Connection header received
0926 14:56:02.573944 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:02.573945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:02.573954 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:02.573998 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:02.574116 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.574856 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.576256 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.576806 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.576865 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.576873 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:02.577019 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 14:56:02.577051 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 14:56:02.577121 9535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.573756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:46303 (local address 127.7.37.194:44771)
0926 14:56:02.573793 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:02.573795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:02.573805 (+ 10us) server_negotiation.cc:407] Connection header received
0926 14:56:02.573882 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:02.573883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:02.573894 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:02.573951 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:02.574097 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.574837 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.576261 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.576792 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.576865 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.576873 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:02.577019 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 14:56:02.577052 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 14:56:02.577145 9132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.573682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:44771 (local address 127.7.37.193:46303)
0926 14:56:02.573733 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:02.573745 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:02.573787 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:02.573962 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:02.573963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:02.573965 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:02.574030 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.574032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.574906 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.574907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.576182 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.576185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.576871 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.576872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.576952 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.576954 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:56:02.577010 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 14:56:02.577056 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":48}
I20260926 14:56:02.577155 9548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.573683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:42517 (local address 127.7.37.193:52231)
0926 14:56:02.573848 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:02.573856 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:02.573898 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:02.574006 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:02.574007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:02.574008 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:02.574063 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.574065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.574913 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.574913 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.576182 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.576185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.576871 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.576872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.576950 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.576954 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:56:02.577010 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 14:56:02.577055 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":82,"thread_start_us":78,"threads_started":1}
I20260926 14:56:02.577451 9333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7459c81afb845a4a2ed01084fc103a1" candidate_uuid: "ce649b409ece4d3483a8c8920e127e61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4af784ae2f34378ab2dafec3ae54481" is_pre_election: true
I20260926 14:56:02.577466 9472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7459c81afb845a4a2ed01084fc103a1" candidate_uuid: "ce649b409ece4d3483a8c8920e127e61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b060492dd71a43b987c33636317ecd17" is_pre_election: true
I20260926 14:56:02.577584 9472 raft_consensus.cc:2454] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce649b409ece4d3483a8c8920e127e61 in term 0.
I20260926 14:56:02.577584 9333 raft_consensus.cc:2454] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce649b409ece4d3483a8c8920e127e61 in term 0.
I20260926 14:56:02.577780 9125 leader_election.cc:302] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b060492dd71a43b987c33636317ecd17, ce649b409ece4d3483a8c8920e127e61; no voters:
I20260926 14:56:02.577898 9547 raft_consensus.cc:2781] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:56:02.577963 9547 raft_consensus.cc:482] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:56:02.577981 9547 raft_consensus.cc:3037] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:02.578601 9547 raft_consensus.cc:504] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.578701 9547 leader_election.cc:288] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [CANDIDATE]: Term 1 election: Requested vote from peers b4af784ae2f34378ab2dafec3ae54481 (127.7.37.194:44771), b060492dd71a43b987c33636317ecd17 (127.7.37.195:42517)
I20260926 14:56:02.578859 9472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7459c81afb845a4a2ed01084fc103a1" candidate_uuid: "ce649b409ece4d3483a8c8920e127e61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b060492dd71a43b987c33636317ecd17"
I20260926 14:56:02.578922 9472 raft_consensus.cc:3037] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:02.579524 9472 raft_consensus.cc:2454] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce649b409ece4d3483a8c8920e127e61 in term 1.
I20260926 14:56:02.579617 9333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7459c81afb845a4a2ed01084fc103a1" candidate_uuid: "ce649b409ece4d3483a8c8920e127e61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4af784ae2f34378ab2dafec3ae54481"
I20260926 14:56:02.579692 9333 raft_consensus.cc:3037] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:02.579701 9125 leader_election.cc:302] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b060492dd71a43b987c33636317ecd17, ce649b409ece4d3483a8c8920e127e61; no voters:
I20260926 14:56:02.579797 9547 raft_consensus.cc:2781] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:02.579928 9547 raft_consensus.cc:686] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [term 1 LEADER]: Becoming Leader. State: Replica: ce649b409ece4d3483a8c8920e127e61, State: Running, Role: LEADER
I20260926 14:56:02.580080 9547 consensus_queue.cc:237] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.580392 9333 raft_consensus.cc:2454] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce649b409ece4d3483a8c8920e127e61 in term 1.
I20260926 14:56:02.580770 9036 catalog_manager.cc:5495] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce649b409ece4d3483a8c8920e127e61 (127.7.37.193). New cstate: current_term: 1 leader_uuid: "ce649b409ece4d3483a8c8920e127e61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } health_report { overall_health: UNKNOWN } } }
W20260926 14:56:02.623494 9240 tablet.cc:2369] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 14:56:02 dist-test-slave-f9xb krb5kdc[8991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434561, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:02.630229 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.624295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45760 (local address 127.7.37.254:42985)
0926 14:56:02.624353 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:02.624354 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:02.624443 (+ 89us) server_negotiation.cc:407] Connection header received
0926 14:56:02.624509 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:02.624510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:02.624517 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:02.624574 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:02.624962 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.625412 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.626102 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.626281 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.626336 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.628158 (+ 1822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:02.628159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:02.628160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:02.628165 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:02.629373 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:02.629694 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:02.629694 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:02.629694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:02.629741 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:02.629886 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:02.629887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:02.629887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:02.629970 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:02.630011 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:02.630116 (+ 105us) server_negotiation.cc:299] Negotiation successful
0926 14:56:02.630174 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 14:56:02.630326 9556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.624212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42985 (local address 127.0.0.1:45760)
0926 14:56:02.624378 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:02.624389 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:02.624451 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:02.624584 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:02.624585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:02.624850 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:02.624893 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.624895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.625477 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.625478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.626022 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.626024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.626342 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.626343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.626392 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.626816 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:02.626820 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:02.628078 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:02.629439 (+ 1361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:02.629446 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:02.629448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:02.629637 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:02.629791 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:02.629791 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:02.629791 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:02.629844 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:02.630012 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:02.630015 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:02.630052 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 14:56:02.630102 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 14:56:02.630220 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":41,"thread_start_us":73,"threads_started":1}
I20260926 14:56:02.632079 9036 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:45760:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 14:56:02.632247 9036 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:56:02.633440 9173 tablet_service.cc:1467] Processing CreateTablet for tablet f1d57c69a78b4a34b1dd01206496ba6e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=394a1c4669c24208ab97ac4d17667229]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 14:56:02.633478 9313 tablet_service.cc:1467] Processing CreateTablet for tablet f1d57c69a78b4a34b1dd01206496ba6e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=394a1c4669c24208ab97ac4d17667229]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 14:56:02.633533 9453 tablet_service.cc:1467] Processing CreateTablet for tablet f1d57c69a78b4a34b1dd01206496ba6e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=394a1c4669c24208ab97ac4d17667229]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 14:56:02.633594 9173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1d57c69a78b4a34b1dd01206496ba6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:02.633608 9313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1d57c69a78b4a34b1dd01206496ba6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:02.633678 9453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1d57c69a78b4a34b1dd01206496ba6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:02.634950 9540 tablet_bootstrap.cc:492] T f1d57c69a78b4a34b1dd01206496ba6e P b4af784ae2f34378ab2dafec3ae54481: Bootstrap starting.
I20260926 14:56:02.635000 9539 tablet_bootstrap.cc:492] T f1d57c69a78b4a34b1dd01206496ba6e P b060492dd71a43b987c33636317ecd17: Bootstrap starting.
I20260926 14:56:02.635428 9539 tablet_bootstrap.cc:654] T f1d57c69a78b4a34b1dd01206496ba6e P b060492dd71a43b987c33636317ecd17: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:02.635489 9540 tablet_bootstrap.cc:654] T f1d57c69a78b4a34b1dd01206496ba6e P b4af784ae2f34378ab2dafec3ae54481: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:02.635828 9545 tablet_bootstrap.cc:492] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61: Bootstrap starting.
I20260926 14:56:02.636030 9539 tablet_bootstrap.cc:492] T f1d57c69a78b4a34b1dd01206496ba6e P b060492dd71a43b987c33636317ecd17: No bootstrap required, opened a new log
I20260926 14:56:02.636051 9540 tablet_bootstrap.cc:492] T f1d57c69a78b4a34b1dd01206496ba6e P b4af784ae2f34378ab2dafec3ae54481: No bootstrap required, opened a new log
I20260926 14:56:02.636073 9539 ts_tablet_manager.cc:1397] T f1d57c69a78b4a34b1dd01206496ba6e P b060492dd71a43b987c33636317ecd17: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 14:56:02.636089 9540 ts_tablet_manager.cc:1397] T f1d57c69a78b4a34b1dd01206496ba6e P b4af784ae2f34378ab2dafec3ae54481: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:02.636224 9540 raft_consensus.cc:348] T f1d57c69a78b4a34b1dd01206496ba6e P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.636219 9539 raft_consensus.cc:348] T f1d57c69a78b4a34b1dd01206496ba6e P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.636281 9540 raft_consensus.cc:374] T f1d57c69a78b4a34b1dd01206496ba6e P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:02.636291 9539 raft_consensus.cc:374] T f1d57c69a78b4a34b1dd01206496ba6e P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:02.636302 9540 raft_consensus.cc:729] T f1d57c69a78b4a34b1dd01206496ba6e P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4af784ae2f34378ab2dafec3ae54481, State: Initialized, Role: FOLLOWER
I20260926 14:56:02.636303 9545 tablet_bootstrap.cc:654] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:02.636312 9539 raft_consensus.cc:729] T f1d57c69a78b4a34b1dd01206496ba6e P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b060492dd71a43b987c33636317ecd17, State: Initialized, Role: FOLLOWER
I20260926 14:56:02.636355 9539 consensus_queue.cc:260] T f1d57c69a78b4a34b1dd01206496ba6e P b060492dd71a43b987c33636317ecd17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.636355 9540 consensus_queue.cc:260] T f1d57c69a78b4a34b1dd01206496ba6e P b4af784ae2f34378ab2dafec3ae54481 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.636445 9540 ts_tablet_manager.cc:1428] T f1d57c69a78b4a34b1dd01206496ba6e P b4af784ae2f34378ab2dafec3ae54481: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:02.636456 9539 ts_tablet_manager.cc:1428] T f1d57c69a78b4a34b1dd01206496ba6e P b060492dd71a43b987c33636317ecd17: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:02.636504 9543 tablet_replica.cc:439] T f1d57c69a78b4a34b1dd01206496ba6e P b4af784ae2f34378ab2dafec3ae54481: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } } }
I20260926 14:56:02.636592 9543 tablet_replica.cc:442] T f1d57c69a78b4a34b1dd01206496ba6e P b4af784ae2f34378ab2dafec3ae54481: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:02.636708 9544 tablet_replica.cc:439] T f1d57c69a78b4a34b1dd01206496ba6e P b060492dd71a43b987c33636317ecd17: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } } }
I20260926 14:56:02.636785 9544 tablet_replica.cc:442] T f1d57c69a78b4a34b1dd01206496ba6e P b060492dd71a43b987c33636317ecd17: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:02.637004 9545 tablet_bootstrap.cc:492] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61: No bootstrap required, opened a new log
I20260926 14:56:02.637055 9545 ts_tablet_manager.cc:1397] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:02.637177 9545 raft_consensus.cc:348] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.637231 9545 raft_consensus.cc:374] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:02.637248 9545 raft_consensus.cc:729] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce649b409ece4d3483a8c8920e127e61, State: Initialized, Role: FOLLOWER
I20260926 14:56:02.637284 9545 consensus_queue.cc:260] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.637356 9545 ts_tablet_manager.cc:1428] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:02.637377 9547 tablet_replica.cc:439] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } } }
I20260926 14:56:02.637434 9547 tablet_replica.cc:442] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61: This TxnStatusTablet replica's current role is: FOLLOWER
W20260926 14:56:02.680752 9520 tablet.cc:2369] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:56:02.848927 9547 raft_consensus.cc:482] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:56:02.849004 9547 raft_consensus.cc:504] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.849157 9547 leader_election.cc:288] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4af784ae2f34378ab2dafec3ae54481 (127.7.37.194:44771), b060492dd71a43b987c33636317ecd17 (127.7.37.195:42517)
I20260926 14:56:02.849330 9472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1d57c69a78b4a34b1dd01206496ba6e" candidate_uuid: "ce649b409ece4d3483a8c8920e127e61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b060492dd71a43b987c33636317ecd17" is_pre_election: true
I20260926 14:56:02.849344 9333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1d57c69a78b4a34b1dd01206496ba6e" candidate_uuid: "ce649b409ece4d3483a8c8920e127e61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4af784ae2f34378ab2dafec3ae54481" is_pre_election: true
I20260926 14:56:02.849408 9472 raft_consensus.cc:2454] T f1d57c69a78b4a34b1dd01206496ba6e P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce649b409ece4d3483a8c8920e127e61 in term 0.
I20260926 14:56:02.849426 9333 raft_consensus.cc:2454] T f1d57c69a78b4a34b1dd01206496ba6e P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce649b409ece4d3483a8c8920e127e61 in term 0.
I20260926 14:56:02.849537 9125 leader_election.cc:302] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b060492dd71a43b987c33636317ecd17, ce649b409ece4d3483a8c8920e127e61; no voters:
I20260926 14:56:02.849632 9547 raft_consensus.cc:2781] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:56:02.849673 9547 raft_consensus.cc:482] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:56:02.849684 9547 raft_consensus.cc:3037] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:02.850293 9547 raft_consensus.cc:504] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.850394 9547 leader_election.cc:288] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 [CANDIDATE]: Term 1 election: Requested vote from peers b4af784ae2f34378ab2dafec3ae54481 (127.7.37.194:44771), b060492dd71a43b987c33636317ecd17 (127.7.37.195:42517)
I20260926 14:56:02.850545 9472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1d57c69a78b4a34b1dd01206496ba6e" candidate_uuid: "ce649b409ece4d3483a8c8920e127e61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b060492dd71a43b987c33636317ecd17"
I20260926 14:56:02.850536 9333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1d57c69a78b4a34b1dd01206496ba6e" candidate_uuid: "ce649b409ece4d3483a8c8920e127e61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4af784ae2f34378ab2dafec3ae54481"
I20260926 14:56:02.850610 9333 raft_consensus.cc:3037] T f1d57c69a78b4a34b1dd01206496ba6e P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:02.850608 9472 raft_consensus.cc:3037] T f1d57c69a78b4a34b1dd01206496ba6e P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:02.851320 9472 raft_consensus.cc:2454] T f1d57c69a78b4a34b1dd01206496ba6e P b060492dd71a43b987c33636317ecd17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce649b409ece4d3483a8c8920e127e61 in term 1.
I20260926 14:56:02.851320 9333 raft_consensus.cc:2454] T f1d57c69a78b4a34b1dd01206496ba6e P b4af784ae2f34378ab2dafec3ae54481 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce649b409ece4d3483a8c8920e127e61 in term 1.
I20260926 14:56:02.851467 9125 leader_election.cc:302] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b060492dd71a43b987c33636317ecd17, ce649b409ece4d3483a8c8920e127e61; no voters:
I20260926 14:56:02.851591 9547 raft_consensus.cc:2781] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:02.851640 9547 raft_consensus.cc:686] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 [term 1 LEADER]: Becoming Leader. State: Replica: ce649b409ece4d3483a8c8920e127e61, State: Running, Role: LEADER
I20260926 14:56:02.851686 9547 consensus_queue.cc:237] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.851940 9549 tablet_replica.cc:439] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61: TxnStatusTablet state changed. Reason: New leader ce649b409ece4d3483a8c8920e127e61. Latest consensus state: current_term: 1 leader_uuid: "ce649b409ece4d3483a8c8920e127e61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } } }
I20260926 14:56:02.852000 9549 tablet_replica.cc:442] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61: This TxnStatusTablet replica's current role is: LEADER
I20260926 14:56:02.852146 9561 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 14:56:02.852242 9561 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 14:56:02.852185 9035 catalog_manager.cc:5495] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce649b409ece4d3483a8c8920e127e61 (127.7.37.193). New cstate: current_term: 1 leader_uuid: "ce649b409ece4d3483a8c8920e127e61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } health_report { overall_health: UNKNOWN } } }
I20260926 14:56:02.861445 9538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.858912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59082 (local address 127.7.37.193:37455)
0926 14:56:02.858955 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:02.858956 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:02.858964 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:02.859046 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:02.859047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:02.859062 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:02.859124 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:02.859238 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.859694 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.860750 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.860923 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.860963 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.861142 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:02.861221 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:02.861292 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:02.861364 (+ 72us) server_negotiation.cc:299] Negotiation successful
0926 14:56:02.861383 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 14:56:02.861469 9556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.858844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:37455 (local address 127.0.0.1:59082)
0926 14:56:02.858927 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:02.858938 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:02.858976 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:02.859131 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:02.859132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:02.859134 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 14:56:02.859181 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.859184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.859776 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.859777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.860673 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.860676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.860977 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.860978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.861049 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.861065 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 14:56:02.861309 (+ 244us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 14:56:02.861319 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 14:56:02.861362 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 14:56:02.861406 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260926 14:56:02.863329 9333 raft_consensus.cc:1261] T f1d57c69a78b4a34b1dd01206496ba6e P b4af784ae2f34378ab2dafec3ae54481 [term 1 FOLLOWER]: Refusing update from remote peer ce649b409ece4d3483a8c8920e127e61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:02.863348 9472 raft_consensus.cc:1261] T f1d57c69a78b4a34b1dd01206496ba6e P b060492dd71a43b987c33636317ecd17 [term 1 FOLLOWER]: Refusing update from remote peer ce649b409ece4d3483a8c8920e127e61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:02.863593 9547 consensus_queue.cc:1035] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:02.863806 9549 consensus_queue.cc:1035] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:02.864857 9543 tablet_replica.cc:439] T f1d57c69a78b4a34b1dd01206496ba6e P b4af784ae2f34378ab2dafec3ae54481: TxnStatusTablet state changed. Reason: New leader ce649b409ece4d3483a8c8920e127e61. Latest consensus state: current_term: 1 leader_uuid: "ce649b409ece4d3483a8c8920e127e61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } } }
I20260926 14:56:02.864934 9543 tablet_replica.cc:442] T f1d57c69a78b4a34b1dd01206496ba6e P b4af784ae2f34378ab2dafec3ae54481: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:02.864874 9544 tablet_replica.cc:439] T f1d57c69a78b4a34b1dd01206496ba6e P b060492dd71a43b987c33636317ecd17: TxnStatusTablet state changed. Reason: New leader ce649b409ece4d3483a8c8920e127e61. Latest consensus state: current_term: 1 leader_uuid: "ce649b409ece4d3483a8c8920e127e61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } } }
I20260926 14:56:02.864961 9544 tablet_replica.cc:442] T f1d57c69a78b4a34b1dd01206496ba6e P b060492dd71a43b987c33636317ecd17: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:02.865813 9549 tablet_replica.cc:439] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce649b409ece4d3483a8c8920e127e61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } } }
I20260926 14:56:02.865895 9549 tablet_replica.cc:442] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61: This TxnStatusTablet replica's current role is: LEADER
I20260926 14:56:02.865938 9549 tablet_replica.cc:439] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce649b409ece4d3483a8c8920e127e61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } } }
I20260926 14:56:02.865972 9549 tablet_replica.cc:442] T f1d57c69a78b4a34b1dd01206496ba6e P ce649b409ece4d3483a8c8920e127e61: This TxnStatusTablet replica's current role is: LEADER
I20260926 14:56:02.866022 9564 mvcc.cc:205] Tried to move back new op lower bound from 7333619969486319616 to 7333619969440866304. Current Snapshot: MvccSnapshot[applied={T|T < 7333619969486319616}]
I20260926 14:56:02.865979 9544 tablet_replica.cc:439] T f1d57c69a78b4a34b1dd01206496ba6e P b060492dd71a43b987c33636317ecd17: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce649b409ece4d3483a8c8920e127e61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } } }
I20260926 14:56:02.866016 9543 tablet_replica.cc:439] T f1d57c69a78b4a34b1dd01206496ba6e P b4af784ae2f34378ab2dafec3ae54481: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce649b409ece4d3483a8c8920e127e61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } } }
I20260926 14:56:02.866058 9544 tablet_replica.cc:442] T f1d57c69a78b4a34b1dd01206496ba6e P b060492dd71a43b987c33636317ecd17: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:02.866077 9543 tablet_replica.cc:442] T f1d57c69a78b4a34b1dd01206496ba6e P b4af784ae2f34378ab2dafec3ae54481: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:02.866523 9566 mvcc.cc:205] Tried to move back new op lower bound from 7333619969486319616 to 7333619969440866304. Current Snapshot: MvccSnapshot[applied={T|T < 7333619969486319616}]
I20260926 14:56:02.866787 9565 mvcc.cc:205] Tried to move back new op lower bound from 7333619969486319616 to 7333619969440866304. Current Snapshot: MvccSnapshot[applied={T|T < 7333619969486319616}]
I20260926 14:56:02.870563 9538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.868481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59096 (local address 127.7.37.193:37455)
0926 14:56:02.868534 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:02.868536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:02.868543 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:02.868561 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:02.868561 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:02.868568 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:02.868608 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:02.868735 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.869291 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.869993 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.870188 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.870237 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.870363 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:02.870390 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:02.870441 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:02.870493 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 14:56:02.870508 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 14:56:02.875001 9538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.872687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:43331 (local address 127.7.37.193:37455)
0926 14:56:02.872717 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:02.872719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:02.872792 (+ 73us) server_negotiation.cc:407] Connection header received
0926 14:56:02.872854 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:02.872855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:02.872862 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:02.872897 (+ 35us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:02.873027 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.873423 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.874450 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.874612 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.874658 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.874827 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:02.874845 (+ 18us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:02.874874 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:02.874927 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 14:56:02.874940 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":18}
I20260926 14:56:02.875093 9576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.872633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:37455 (local address 127.7.37.193:43331)
0926 14:56:02.872741 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:02.872750 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:02.872804 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:02.872909 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:02.872910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:02.872912 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 14:56:02.872968 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.872969 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.873476 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.873476 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.874384 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.874386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.874670 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.874671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.874741 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.874753 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 14:56:02.874877 (+ 124us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 14:56:02.874878 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 14:56:02.874923 (+ 45us) client_negotiation.cc:240] Negotiation successful
0926 14:56:02.874959 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":62,"threads_started":1}
I20260926 14:56:02.877022 9333 raft_consensus.cc:1261] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481 [term 1 FOLLOWER]: Refusing update from remote peer ce649b409ece4d3483a8c8920e127e61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:02.877020 9472 raft_consensus.cc:1261] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17 [term 1 FOLLOWER]: Refusing update from remote peer ce649b409ece4d3483a8c8920e127e61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:02.877216 9547 consensus_queue.cc:1035] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:02.877295 9569 consensus_queue.cc:1035] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:02.886835 9034 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:02.886915 9034 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:02.887513 9034 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a: Sending DeleteTablet for 3 replicas of tablet a7459c81afb845a4a2ed01084fc103a1
I20260926 14:56:02.887775 9313 tablet_service.cc:1514] Processing DeleteTablet for tablet a7459c81afb845a4a2ed01084fc103a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:02 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:52466
I20260926 14:56:02.887777 9453 tablet_service.cc:1514] Processing DeleteTablet for tablet a7459c81afb845a4a2ed01084fc103a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:02 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:53488
I20260926 14:56:02.887814 9173 tablet_service.cc:1514] Processing DeleteTablet for tablet a7459c81afb845a4a2ed01084fc103a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:02 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:59074
I20260926 14:56:02.887964 9580 tablet_replica.cc:330] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17: stopping tablet replica
I20260926 14:56:02.888055 9580 raft_consensus.cc:2229] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:02.888134 9580 raft_consensus.cc:2258] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:02.888203 9034 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:56:02.888322 9034 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:56:02.889376 9173 tablet_service.cc:1467] Processing CreateTablet for tablet 44153159d73a46d788b77ebf4a7a1c1a (DEFAULT_TABLE table=test-table [id=af8a8c26cb8f42a0a6da2f43863956c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:02.889379 9453 tablet_service.cc:1467] Processing CreateTablet for tablet 44153159d73a46d788b77ebf4a7a1c1a (DEFAULT_TABLE table=test-table [id=af8a8c26cb8f42a0a6da2f43863956c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:02.889401 9313 tablet_service.cc:1467] Processing CreateTablet for tablet 44153159d73a46d788b77ebf4a7a1c1a (DEFAULT_TABLE table=test-table [id=af8a8c26cb8f42a0a6da2f43863956c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:02.889474 9313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44153159d73a46d788b77ebf4a7a1c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:02.889474 9173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44153159d73a46d788b77ebf4a7a1c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:02.889479 9453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44153159d73a46d788b77ebf4a7a1c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:02.890314 9539 tablet_bootstrap.cc:492] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17: Bootstrap starting.
I20260926 14:56:02.890615 9540 tablet_bootstrap.cc:492] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481: Bootstrap starting.
I20260926 14:56:02.890671 9539 tablet_bootstrap.cc:654] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:02.890928 9540 tablet_bootstrap.cc:654] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:02.890980 9545 tablet_bootstrap.cc:492] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61: Bootstrap starting.
I20260926 14:56:02.891075 9581 tablet_replica.cc:330] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481: stopping tablet replica
I20260926 14:56:02.891198 9581 raft_consensus.cc:2229] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:02.891280 9581 raft_consensus.cc:2258] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:02.891383 9545 tablet_bootstrap.cc:654] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:02.891490 9582 tablet_replica.cc:330] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61: stopping tablet replica
I20260926 14:56:02.891575 9582 raft_consensus.cc:2229] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:56:02.891669 9582 raft_consensus.cc:2258] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:02.891676 9581 ts_tablet_manager.cc:1905] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:02.891987 9539 tablet_bootstrap.cc:492] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17: No bootstrap required, opened a new log
I20260926 14:56:02.892019 9545 tablet_bootstrap.cc:492] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61: No bootstrap required, opened a new log
I20260926 14:56:02.892037 9539 ts_tablet_manager.cc:1397] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 14:56:02.892050 9545 ts_tablet_manager.cc:1397] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:02.892169 9545 raft_consensus.cc:348] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.892182 9539 raft_consensus.cc:348] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.892231 9545 raft_consensus.cc:374] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:02.892231 9539 raft_consensus.cc:374] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:02.892251 9545 raft_consensus.cc:729] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce649b409ece4d3483a8c8920e127e61, State: Initialized, Role: FOLLOWER
I20260926 14:56:02.892251 9539 raft_consensus.cc:729] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b060492dd71a43b987c33636317ecd17, State: Initialized, Role: FOLLOWER
I20260926 14:56:02.892285 9539 consensus_queue.cc:260] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.892285 9545 consensus_queue.cc:260] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.892360 9545 ts_tablet_manager.cc:1428] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:02.892359 9539 ts_tablet_manager.cc:1428] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:02.892998 9540 tablet_bootstrap.cc:492] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481: No bootstrap required, opened a new log
I20260926 14:56:02.893045 9540 ts_tablet_manager.cc:1397] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:56:02.893174 9540 raft_consensus.cc:348] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.893222 9582 ts_tablet_manager.cc:1905] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:02.893241 9540 raft_consensus.cc:374] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:02.893268 9540 raft_consensus.cc:729] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4af784ae2f34378ab2dafec3ae54481, State: Initialized, Role: FOLLOWER
I20260926 14:56:02.893301 9540 consensus_queue.cc:260] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.893380 9540 ts_tablet_manager.cc:1428] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:02.893386 9581 ts_tablet_manager.cc:1918] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 14:56:02.893532 9581 log.cc:1196] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/a7459c81afb845a4a2ed01084fc103a1
I20260926 14:56:02.893811 9581 ts_tablet_manager.cc:1939] T a7459c81afb845a4a2ed01084fc103a1 P b4af784ae2f34378ab2dafec3ae54481: Deleting consensus metadata
I20260926 14:56:02.893934 9580 ts_tablet_manager.cc:1905] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:02.894338 9020 catalog_manager.cc:4841] TS b4af784ae2f34378ab2dafec3ae54481 (127.7.37.194:44771): tablet a7459c81afb845a4a2ed01084fc103a1 (table test-table [id=0a3f696841c84041bea0bbfa22a3e518]) successfully deleted
I20260926 14:56:02.894584 9582 ts_tablet_manager.cc:1918] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 14:56:02.894660 9582 log.cc:1196] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/wal/wals/a7459c81afb845a4a2ed01084fc103a1
I20260926 14:56:02.894884 9582 ts_tablet_manager.cc:1939] T a7459c81afb845a4a2ed01084fc103a1 P ce649b409ece4d3483a8c8920e127e61: Deleting consensus metadata
I20260926 14:56:02.895217 9020 catalog_manager.cc:4841] TS ce649b409ece4d3483a8c8920e127e61 (127.7.37.193:37455): tablet a7459c81afb845a4a2ed01084fc103a1 (table test-table [id=0a3f696841c84041bea0bbfa22a3e518]) successfully deleted
I20260926 14:56:02.895495 9580 ts_tablet_manager.cc:1918] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 14:56:02.895546 9580 log.cc:1196] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/a7459c81afb845a4a2ed01084fc103a1
I20260926 14:56:02.895696 9580 ts_tablet_manager.cc:1939] T a7459c81afb845a4a2ed01084fc103a1 P b060492dd71a43b987c33636317ecd17: Deleting consensus metadata
I20260926 14:56:02.896039 9021 catalog_manager.cc:4841] TS b060492dd71a43b987c33636317ecd17 (127.7.37.195:42517): tablet a7459c81afb845a4a2ed01084fc103a1 (table test-table [id=0a3f696841c84041bea0bbfa22a3e518]) successfully deleted
I20260926 14:56:02.995540 9544 raft_consensus.cc:482] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:56:02.995626 9544 raft_consensus.cc:504] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:02.995908 9544 leader_election.cc:288] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce649b409ece4d3483a8c8920e127e61 (127.7.37.193:37455), b4af784ae2f34378ab2dafec3ae54481 (127.7.37.194:44771)
I20260926 14:56:02.998947 9538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.996115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:53937 (local address 127.7.37.193:37455)
0926 14:56:02.996165 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:02.996167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:02.996252 (+ 85us) server_negotiation.cc:407] Connection header received
0926 14:56:02.996350 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:02.996350 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:02.996360 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:02.996388 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:02.996605 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.997335 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.998309 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.998660 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.998727 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.998737 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:02.998848 (+ 111us) server_negotiation.cc:299] Negotiation successful
0926 14:56:02.998882 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 14:56:02.998963 9586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.996068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:37455 (local address 127.7.37.195:53937)
0926 14:56:02.996186 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:02.996197 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:02.996279 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:02.996448 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:02.996449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:02.996451 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:02.996535 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.996537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.997443 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.997444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.998227 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.998229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.998722 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.998723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.998790 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.998794 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:56:02.998839 (+ 45us) client_negotiation.cc:240] Negotiation successful
0926 14:56:02.998879 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":51,"thread_start_us":59,"threads_started":1}
I20260926 14:56:02.999290 9193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44153159d73a46d788b77ebf4a7a1c1a" candidate_uuid: "b060492dd71a43b987c33636317ecd17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce649b409ece4d3483a8c8920e127e61" is_pre_election: true
I20260926 14:56:02.999405 9193 raft_consensus.cc:2454] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b060492dd71a43b987c33636317ecd17 in term 0.
I20260926 14:56:02.999544 9535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.996291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:38249 (local address 127.7.37.194:44771)
0926 14:56:02.996330 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:02.996331 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:02.996364 (+ 33us) server_negotiation.cc:407] Connection header received
0926 14:56:02.996425 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:02.996426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:02.996433 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:02.996474 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:02.996637 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.997335 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.998686 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.999187 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.999249 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.999257 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:02.999444 (+ 187us) server_negotiation.cc:299] Negotiation successful
0926 14:56:02.999484 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 14:56:02.999567 9404 leader_election.cc:302] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b060492dd71a43b987c33636317ecd17, ce649b409ece4d3483a8c8920e127e61; no voters:
I20260926 14:56:02.999704 9544 raft_consensus.cc:2781] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:56:02.999774 9544 raft_consensus.cc:482] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:56:02.999792 9587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:02.996229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:44771 (local address 127.7.37.195:38249)
0926 14:56:02.996317 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:02.996327 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:02.996377 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:02.996487 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:02.996487 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:02.996489 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:02.996547 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.996549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.997411 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.997412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.998599 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:02.998601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:02.999299 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:02.999300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:02.999371 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:02.999374 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:02.999626 (+ 252us) client_negotiation.cc:240] Negotiation successful
0926 14:56:02.999674 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":2,"thread_start_us":30,"threads_started":1}
I20260926 14:56:02.999796 9544 raft_consensus.cc:3037] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:03.000047 9333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44153159d73a46d788b77ebf4a7a1c1a" candidate_uuid: "b060492dd71a43b987c33636317ecd17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4af784ae2f34378ab2dafec3ae54481" is_pre_election: true
I20260926 14:56:03.000136 9333 raft_consensus.cc:2454] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b060492dd71a43b987c33636317ecd17 in term 0.
I20260926 14:56:03.000631 9544 raft_consensus.cc:504] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:03.000732 9544 leader_election.cc:288] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [CANDIDATE]: Term 1 election: Requested vote from peers ce649b409ece4d3483a8c8920e127e61 (127.7.37.193:37455), b4af784ae2f34378ab2dafec3ae54481 (127.7.37.194:44771)
I20260926 14:56:03.000861 9193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44153159d73a46d788b77ebf4a7a1c1a" candidate_uuid: "b060492dd71a43b987c33636317ecd17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce649b409ece4d3483a8c8920e127e61"
I20260926 14:56:03.000931 9193 raft_consensus.cc:3037] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:03.000895 9333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44153159d73a46d788b77ebf4a7a1c1a" candidate_uuid: "b060492dd71a43b987c33636317ecd17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4af784ae2f34378ab2dafec3ae54481"
I20260926 14:56:03.000957 9333 raft_consensus.cc:3037] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:03.001605 9193 raft_consensus.cc:2454] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b060492dd71a43b987c33636317ecd17 in term 1.
I20260926 14:56:03.001605 9333 raft_consensus.cc:2454] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b060492dd71a43b987c33636317ecd17 in term 1.
I20260926 14:56:03.001753 9404 leader_election.cc:302] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b060492dd71a43b987c33636317ecd17, ce649b409ece4d3483a8c8920e127e61; no voters:
I20260926 14:56:03.001842 9544 raft_consensus.cc:2781] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:03.001935 9544 raft_consensus.cc:686] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [term 1 LEADER]: Becoming Leader. State: Replica: b060492dd71a43b987c33636317ecd17, State: Running, Role: LEADER
I20260926 14:56:03.001995 9544 consensus_queue.cc:237] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:03.002583 9034 catalog_manager.cc:5495] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 reported cstate change: term changed from 0 to 1, leader changed from <none> to b060492dd71a43b987c33636317ecd17 (127.7.37.195). New cstate: current_term: 1 leader_uuid: "b060492dd71a43b987c33636317ecd17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } health_report { overall_health: HEALTHY } } }
I20260926 14:56:03.009557 9537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:03.007496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53500 (local address 127.7.37.195:42517)
0926 14:56:03.007588 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:03.007590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:03.007595 (+ 5us) server_negotiation.cc:407] Connection header received
0926 14:56:03.007615 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:03.007616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:03.007631 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:03.007681 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:03.007802 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:03.008237 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:03.008902 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:03.009100 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:03.009149 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:03.009277 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:03.009370 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:03.009426 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:03.009470 (+ 44us) server_negotiation.cc:299] Negotiation successful
0926 14:56:03.009487 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 14:56:03.013782 9538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:03.011787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:43179 (local address 127.7.37.193:37455)
0926 14:56:03.011824 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:03.011826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:03.011863 (+ 37us) server_negotiation.cc:407] Connection header received
0926 14:56:03.011928 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:03.011928 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:03.011936 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:03.011975 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:03.012094 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:03.012517 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:03.013227 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:03.013412 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:03.013461 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:03.013590 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:03.013620 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:03.013660 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:03.013706 (+ 46us) server_negotiation.cc:299] Negotiation successful
0926 14:56:03.013721 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 14:56:03.013820 9592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:03.011645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:37455 (local address 127.7.37.195:43179)
0926 14:56:03.011807 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:03.011818 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:03.011877 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:03.011976 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:03.011976 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:03.011977 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 14:56:03.012027 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:03.012029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:03.012606 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:03.012607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:03.013158 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:03.013159 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:03.013466 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:03.013467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:03.013516 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:03.013529 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 14:56:03.013666 (+ 137us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 14:56:03.013667 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 14:56:03.013705 (+ 38us) client_negotiation.cc:240] Negotiation successful
0926 14:56:03.013737 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260926 14:56:03.015780 9333 raft_consensus.cc:1261] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481 [term 1 FOLLOWER]: Refusing update from remote peer b060492dd71a43b987c33636317ecd17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:03.015908 9193 raft_consensus.cc:1261] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61 [term 1 FOLLOWER]: Refusing update from remote peer b060492dd71a43b987c33636317ecd17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:03.016041 9588 consensus_queue.cc:1035] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:03.016160 9544 consensus_queue.cc:1035] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:03.022601 9537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:03.020443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:45081 (local address 127.7.37.195:42517)
0926 14:56:03.020473 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:03.020475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:03.020483 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:03.020502 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:03.020503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:03.020512 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:03.020565 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:03.020701 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:03.021123 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:03.022070 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:03.022242 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:03.022291 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:03.022419 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:03.022438 (+ 19us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:03.022467 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:03.022530 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 14:56:03.022544 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260926 14:56:03.022636 9576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:03.020357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:42517 (local address 127.7.37.193:45081)
0926 14:56:03.020425 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:03.020433 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:03.020461 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:03.020604 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:03.020605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:03.020607 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 14:56:03.020644 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:03.020645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:03.021189 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:03.021190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:03.021993 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:03.021995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:03.022304 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:03.022305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:03.022356 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:03.022361 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 14:56:03.022471 (+ 110us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 14:56:03.022473 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:56:03.022526 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 14:56:03.022564 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
W20260926 14:56:03.023154 9565 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 14:56:04.541198 9035 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:04.541290 9035 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:04.541949 9035 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a: Sending DeleteTablet for 3 replicas of tablet 44153159d73a46d788b77ebf4a7a1c1a
I20260926 14:56:04.542240 9173 tablet_service.cc:1514] Processing DeleteTablet for tablet 44153159d73a46d788b77ebf4a7a1c1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:04 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:59074
I20260926 14:56:04.542258 9453 tablet_service.cc:1514] Processing DeleteTablet for tablet 44153159d73a46d788b77ebf4a7a1c1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:04 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:53488
I20260926 14:56:04.542255 9313 tablet_service.cc:1514] Processing DeleteTablet for tablet 44153159d73a46d788b77ebf4a7a1c1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:04 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:52466
I20260926 14:56:04.542435 9620 tablet_replica.cc:330] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17: stopping tablet replica
I20260926 14:56:04.542479 9620 raft_consensus.cc:2229] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:56:04.542544 9620 raft_consensus.cc:2258] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:04.542743 9620 ts_tablet_manager.cc:1905] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:04.544250 9620 ts_tablet_manager.cc:1918] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 14:56:04.544310 9620 log.cc:1196] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/44153159d73a46d788b77ebf4a7a1c1a
I20260926 14:56:04.544548 9620 ts_tablet_manager.cc:1939] T 44153159d73a46d788b77ebf4a7a1c1a P b060492dd71a43b987c33636317ecd17: Deleting consensus metadata
I20260926 14:56:04.544744 9622 tablet_replica.cc:330] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61: stopping tablet replica
I20260926 14:56:04.544821 9622 raft_consensus.cc:2229] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:04.544870 9622 raft_consensus.cc:2258] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:04.544972 9621 tablet_replica.cc:330] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481: stopping tablet replica
I20260926 14:56:04.545029 9621 raft_consensus.cc:2229] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:04.545076 9621 raft_consensus.cc:2258] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:04.545195 9021 catalog_manager.cc:4841] TS b060492dd71a43b987c33636317ecd17 (127.7.37.195:42517): tablet 44153159d73a46d788b77ebf4a7a1c1a (table test-table [id=af8a8c26cb8f42a0a6da2f43863956c9]) successfully deleted
I20260926 14:56:04.545264 9622 ts_tablet_manager.cc:1905] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:04.546411 9622 ts_tablet_manager.cc:1918] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 14:56:04.546459 9622 log.cc:1196] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/wal/wals/44153159d73a46d788b77ebf4a7a1c1a
I20260926 14:56:04.546638 9622 ts_tablet_manager.cc:1939] T 44153159d73a46d788b77ebf4a7a1c1a P ce649b409ece4d3483a8c8920e127e61: Deleting consensus metadata
I20260926 14:56:04.547043 9020 catalog_manager.cc:4841] TS ce649b409ece4d3483a8c8920e127e61 (127.7.37.193:37455): tablet 44153159d73a46d788b77ebf4a7a1c1a (table test-table [id=af8a8c26cb8f42a0a6da2f43863956c9]) successfully deleted
I20260926 14:56:04.547134 9621 ts_tablet_manager.cc:1905] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:04.548177 9621 ts_tablet_manager.cc:1918] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 14:56:04.548223 9621 log.cc:1196] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/44153159d73a46d788b77ebf4a7a1c1a
I20260926 14:56:04.548350 9621 ts_tablet_manager.cc:1939] T 44153159d73a46d788b77ebf4a7a1c1a P b4af784ae2f34378ab2dafec3ae54481: Deleting consensus metadata
I20260926 14:56:04.548784 9020 catalog_manager.cc:4841] TS b4af784ae2f34378ab2dafec3ae54481 (127.7.37.194:44771): tablet 44153159d73a46d788b77ebf4a7a1c1a (table test-table [id=af8a8c26cb8f42a0a6da2f43863956c9]) successfully deleted
Sep 26 14:56:04 dist-test-slave-f9xb krb5kdc[8991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434564, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 14:56:04 dist-test-slave-f9xb krb5kdc[8991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434564, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:04.573907 9630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:04.559359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45766 (local address 127.7.37.254:42985)
0926 14:56:04.559478 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:04.559480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:04.559487 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:04.559530 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:04.559531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:04.559540 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:04.559600 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:04.560129 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.560629 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.561319 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.561541 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.561585 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:04.571628 (+ 10043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:04.571629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:04.571630 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:04.571637 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:04.572940 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:04.573234 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:04.573235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:04.573236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:04.573286 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:04.573441 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:04.573441 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:04.573442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:04.573548 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:04.573633 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:04.573739 (+ 106us) server_negotiation.cc:299] Negotiation successful
0926 14:56:04.573810 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":68,"threads_started":1}
I20260926 14:56:04.575121 9035 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:56:04.575251 9035 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:56:04.576370 9313 tablet_service.cc:1467] Processing CreateTablet for tablet 68801f4d16a743079e509f027f99c3af (DEFAULT_TABLE table=test-table [id=034ff454b8cb4d21884abd7a5518f754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:04.576376 9173 tablet_service.cc:1467] Processing CreateTablet for tablet 68801f4d16a743079e509f027f99c3af (DEFAULT_TABLE table=test-table [id=034ff454b8cb4d21884abd7a5518f754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:04.576402 9453 tablet_service.cc:1467] Processing CreateTablet for tablet 68801f4d16a743079e509f027f99c3af (DEFAULT_TABLE table=test-table [id=034ff454b8cb4d21884abd7a5518f754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:04.576483 9313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68801f4d16a743079e509f027f99c3af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:04.576502 9173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68801f4d16a743079e509f027f99c3af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:04.576501 9453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68801f4d16a743079e509f027f99c3af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:04.577437 9631 tablet_bootstrap.cc:492] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481: Bootstrap starting.
I20260926 14:56:04.577759 9631 tablet_bootstrap.cc:654] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:04.577862 9633 tablet_bootstrap.cc:492] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17: Bootstrap starting.
I20260926 14:56:04.578173 9633 tablet_bootstrap.cc:654] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:04.578328 9631 tablet_bootstrap.cc:492] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481: No bootstrap required, opened a new log
I20260926 14:56:04.578375 9631 ts_tablet_manager.cc:1397] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:04.578450 9632 tablet_bootstrap.cc:492] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61: Bootstrap starting.
I20260926 14:56:04.578507 9631 raft_consensus.cc:348] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:04.578583 9631 raft_consensus.cc:374] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:04.578603 9631 raft_consensus.cc:729] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4af784ae2f34378ab2dafec3ae54481, State: Initialized, Role: FOLLOWER
I20260926 14:56:04.578670 9631 consensus_queue.cc:260] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:04.578753 9632 tablet_bootstrap.cc:654] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:04.578770 9633 tablet_bootstrap.cc:492] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17: No bootstrap required, opened a new log
I20260926 14:56:04.578807 9633 ts_tablet_manager.cc:1397] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:04.578830 9631 ts_tablet_manager.cc:1428] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:04.578915 9633 raft_consensus.cc:348] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:04.578982 9633 raft_consensus.cc:374] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:04.579000 9633 raft_consensus.cc:729] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b060492dd71a43b987c33636317ecd17, State: Initialized, Role: FOLLOWER
I20260926 14:56:04.579046 9633 consensus_queue.cc:260] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:04.579136 9633 ts_tablet_manager.cc:1428] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:04.579372 9632 tablet_bootstrap.cc:492] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61: No bootstrap required, opened a new log
I20260926 14:56:04.579454 9632 ts_tablet_manager.cc:1397] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:04.579593 9632 raft_consensus.cc:348] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:04.579663 9632 raft_consensus.cc:374] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:04.579682 9632 raft_consensus.cc:729] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce649b409ece4d3483a8c8920e127e61, State: Initialized, Role: FOLLOWER
I20260926 14:56:04.579726 9632 consensus_queue.cc:260] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:04.579831 9632 ts_tablet_manager.cc:1428] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:04.723073 9636 raft_consensus.cc:482] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:56:04.723170 9636 raft_consensus.cc:504] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:04.723474 9636 leader_election.cc:288] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce649b409ece4d3483a8c8920e127e61 (127.7.37.193:37455), b060492dd71a43b987c33636317ecd17 (127.7.37.195:42517)
I20260926 14:56:04.726784 9640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:04.723690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:47883 (local address 127.7.37.193:37455)
0926 14:56:04.723833 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:04.723835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:04.723884 (+ 49us) server_negotiation.cc:407] Connection header received
0926 14:56:04.723962 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:04.723963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:04.723973 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:04.724030 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:04.724184 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.724859 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.726109 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.726475 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.726535 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:04.726545 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:04.726681 (+ 136us) server_negotiation.cc:299] Negotiation successful
0926 14:56:04.726720 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260926 14:56:04.726794 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:04.723623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:37455 (local address 127.7.37.194:47883)
0926 14:56:04.723830 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:04.723841 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:04.723900 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:04.724039 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:04.724040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:04.724042 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:04.724128 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:04.724130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.724961 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.724962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:04.726021 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:04.726023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.726543 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.726544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:04.726621 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:04.726625 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:56:04.726668 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 14:56:04.726706 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":8,"thread_start_us":142,"threads_started":1}
I20260926 14:56:04.727061 9193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68801f4d16a743079e509f027f99c3af" candidate_uuid: "b4af784ae2f34378ab2dafec3ae54481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce649b409ece4d3483a8c8920e127e61" is_pre_election: true
I20260926 14:56:04.727147 9193 raft_consensus.cc:2454] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4af784ae2f34378ab2dafec3ae54481 in term 0.
I20260926 14:56:04.727191 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:04.723761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:59391 (local address 127.7.37.195:42517)
0926 14:56:04.723882 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:04.723883 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:04.723891 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:04.723966 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:04.723967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:04.723977 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:04.724037 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:04.724187 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.724923 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.726383 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.726747 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.726805 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:04.726812 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:04.727085 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 14:56:04.727127 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
I20260926 14:56:04.727301 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:04.723655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:42517 (local address 127.7.37.194:59391)
0926 14:56:04.723837 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:04.723848 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:04.723901 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:04.724047 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:04.724048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:04.724050 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:04.724132 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:04.724133 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.725038 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.725038 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:04.726301 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:04.726304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.726814 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.726815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:04.727022 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:04.727026 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:56:04.727072 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 14:56:04.727106 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":4,"thread_start_us":111,"threads_started":1}
I20260926 14:56:04.727319 9264 leader_election.cc:302] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4af784ae2f34378ab2dafec3ae54481, ce649b409ece4d3483a8c8920e127e61; no voters:
I20260926 14:56:04.727429 9636 raft_consensus.cc:2781] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:56:04.727464 9636 raft_consensus.cc:482] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:56:04.727483 9636 raft_consensus.cc:3037] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:04.727499 9472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68801f4d16a743079e509f027f99c3af" candidate_uuid: "b4af784ae2f34378ab2dafec3ae54481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b060492dd71a43b987c33636317ecd17" is_pre_election: true
I20260926 14:56:04.727574 9472 raft_consensus.cc:2454] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4af784ae2f34378ab2dafec3ae54481 in term 0.
I20260926 14:56:04.728161 9636 raft_consensus.cc:504] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:04.728268 9636 leader_election.cc:288] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [CANDIDATE]: Term 1 election: Requested vote from peers ce649b409ece4d3483a8c8920e127e61 (127.7.37.193:37455), b060492dd71a43b987c33636317ecd17 (127.7.37.195:42517)
I20260926 14:56:04.728390 9472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68801f4d16a743079e509f027f99c3af" candidate_uuid: "b4af784ae2f34378ab2dafec3ae54481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b060492dd71a43b987c33636317ecd17"
I20260926 14:56:04.728456 9472 raft_consensus.cc:3037] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:04.728470 9193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68801f4d16a743079e509f027f99c3af" candidate_uuid: "b4af784ae2f34378ab2dafec3ae54481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce649b409ece4d3483a8c8920e127e61"
I20260926 14:56:04.728547 9193 raft_consensus.cc:3037] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:04.729022 9472 raft_consensus.cc:2454] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4af784ae2f34378ab2dafec3ae54481 in term 1.
I20260926 14:56:04.729169 9193 raft_consensus.cc:2454] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4af784ae2f34378ab2dafec3ae54481 in term 1.
I20260926 14:56:04.729182 9265 leader_election.cc:302] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b060492dd71a43b987c33636317ecd17, b4af784ae2f34378ab2dafec3ae54481; no voters:
I20260926 14:56:04.729251 9636 raft_consensus.cc:2781] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:04.729372 9636 raft_consensus.cc:686] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [term 1 LEADER]: Becoming Leader. State: Replica: b4af784ae2f34378ab2dafec3ae54481, State: Running, Role: LEADER
I20260926 14:56:04.729435 9636 consensus_queue.cc:237] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:04.730067 9034 catalog_manager.cc:5495] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4af784ae2f34378ab2dafec3ae54481 (127.7.37.194). New cstate: current_term: 1 leader_uuid: "b4af784ae2f34378ab2dafec3ae54481" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } health_report { overall_health: UNKNOWN } } }
I20260926 14:56:04.761128 9645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:04.758896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52470 (local address 127.7.37.194:44771)
0926 14:56:04.759032 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:04.759033 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:04.759040 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:04.759072 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:04.759072 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:04.759091 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:04.759160 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:04.759255 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.759784 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.760469 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.760673 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.760716 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:04.760847 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:04.760974 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:04.761029 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:04.761046 (+ 17us) server_negotiation.cc:299] Negotiation successful
0926 14:56:04.761061 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
I20260926 14:56:04.765256 9640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:04.763224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:38709 (local address 127.7.37.193:37455)
0926 14:56:04.763274 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:04.763276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:04.763283 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:04.763305 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:04.763306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:04.763316 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:04.763359 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:04.763462 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.763957 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.764706 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.764879 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.764918 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:04.765083 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:04.765112 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:04.765143 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:04.765188 (+ 45us) server_negotiation.cc:299] Negotiation successful
0926 14:56:04.765203 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 14:56:04.765291 9648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:04.763090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:37455 (local address 127.7.37.194:38709)
0926 14:56:04.763228 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:04.763238 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:04.763275 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:04.763365 (+ 90us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:04.763366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:04.763367 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 14:56:04.763418 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:04.763419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.764024 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.764025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:04.764640 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:04.764642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.764952 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.764952 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:04.765007 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:04.765019 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 14:56:04.765154 (+ 135us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 14:56:04.765155 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 14:56:04.765184 (+ 29us) client_negotiation.cc:240] Negotiation successful
0926 14:56:04.765214 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260926 14:56:04.767149 9193 raft_consensus.cc:1261] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61 [term 1 FOLLOWER]: Refusing update from remote peer b4af784ae2f34378ab2dafec3ae54481: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:04.767190 9472 raft_consensus.cc:1261] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17 [term 1 FOLLOWER]: Refusing update from remote peer b4af784ae2f34378ab2dafec3ae54481: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:04.767412 9636 consensus_queue.cc:1035] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:04.767545 9642 consensus_queue.cc:1035] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:04.773653 9645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:04.771500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:55241 (local address 127.7.37.194:44771)
0926 14:56:04.771538 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:04.771540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:04.771677 (+ 137us) server_negotiation.cc:407] Connection header received
0926 14:56:04.771761 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:04.771762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:04.771772 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:04.771816 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:04.771892 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.772319 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.772991 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.773206 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.773263 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:04.773445 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:04.773476 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:04.773515 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:04.773568 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 14:56:04.773587 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:56:04.773686 9653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:04.771473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:44771 (local address 127.7.37.193:55241)
0926 14:56:04.771625 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:04.771636 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:04.771688 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:04.771817 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:04.771818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:04.771820 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 14:56:04.771858 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:04.771859 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.772366 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.772366 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:04.772937 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:04.772939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:04.773314 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:04.773315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:04.773377 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:04.773382 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 14:56:04.773526 (+ 144us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 14:56:04.773528 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:56:04.773565 (+ 37us) client_negotiation.cc:240] Negotiation successful
0926 14:56:04.773617 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":80,"threads_started":1}
I20260926 14:56:04.779906 9035 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:04.779989 9035 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:04.780527 9035 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a: Sending DeleteTablet for 3 replicas of tablet 68801f4d16a743079e509f027f99c3af
I20260926 14:56:04.780737 9313 tablet_service.cc:1514] Processing DeleteTablet for tablet 68801f4d16a743079e509f027f99c3af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:04 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:52466
I20260926 14:56:04.780740 9453 tablet_service.cc:1514] Processing DeleteTablet for tablet 68801f4d16a743079e509f027f99c3af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:04 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:53488
I20260926 14:56:04.780793 9173 tablet_service.cc:1514] Processing DeleteTablet for tablet 68801f4d16a743079e509f027f99c3af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:04 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:59074
I20260926 14:56:04.780818 9621 tablet_replica.cc:330] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481: stopping tablet replica
I20260926 14:56:04.780862 9621 raft_consensus.cc:2229] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:56:04.780865 9620 tablet_replica.cc:330] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17: stopping tablet replica
I20260926 14:56:04.780912 9620 raft_consensus.cc:2229] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:04.780928 9621 raft_consensus.cc:2258] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:04.780944 9620 raft_consensus.cc:2258] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:04.781004 9622 tablet_replica.cc:330] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61: stopping tablet replica
I20260926 14:56:04.781061 9622 raft_consensus.cc:2229] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:04.781098 9622 raft_consensus.cc:2258] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:04.781324 9621 ts_tablet_manager.cc:1905] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:04.781448 9622 ts_tablet_manager.cc:1905] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:04.781445 9620 ts_tablet_manager.cc:1905] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:04.782789 9035 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 14:56:04.782894 9621 ts_tablet_manager.cc:1918] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 14:56:04.782904 9622 ts_tablet_manager.cc:1918] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
W20260926 14:56:04.782919 9035 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:56:04.782918 9620 ts_tablet_manager.cc:1918] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 14:56:04.782943 9621 log.cc:1196] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/68801f4d16a743079e509f027f99c3af
I20260926 14:56:04.782945 9622 log.cc:1196] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-0/wal/wals/68801f4d16a743079e509f027f99c3af
I20260926 14:56:04.782958 9620 log.cc:1196] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/68801f4d16a743079e509f027f99c3af
I20260926 14:56:04.783141 9620 ts_tablet_manager.cc:1939] T 68801f4d16a743079e509f027f99c3af P b060492dd71a43b987c33636317ecd17: Deleting consensus metadata
I20260926 14:56:04.783165 9622 ts_tablet_manager.cc:1939] T 68801f4d16a743079e509f027f99c3af P ce649b409ece4d3483a8c8920e127e61: Deleting consensus metadata
I20260926 14:56:04.783170 9621 ts_tablet_manager.cc:1939] T 68801f4d16a743079e509f027f99c3af P b4af784ae2f34378ab2dafec3ae54481: Deleting consensus metadata
I20260926 14:56:04.783428 9021 catalog_manager.cc:4841] TS b060492dd71a43b987c33636317ecd17 (127.7.37.195:42517): tablet 68801f4d16a743079e509f027f99c3af (table test-table [id=034ff454b8cb4d21884abd7a5518f754]) successfully deleted
I20260926 14:56:04.783493 9020 catalog_manager.cc:4841] TS b4af784ae2f34378ab2dafec3ae54481 (127.7.37.194:44771): tablet 68801f4d16a743079e509f027f99c3af (table test-table [id=034ff454b8cb4d21884abd7a5518f754]) successfully deleted
I20260926 14:56:04.783524 9020 catalog_manager.cc:4841] TS ce649b409ece4d3483a8c8920e127e61 (127.7.37.193:37455): tablet 68801f4d16a743079e509f027f99c3af (table test-table [id=034ff454b8cb4d21884abd7a5518f754]) successfully deleted
I20260926 14:56:04.784325 9173 tablet_service.cc:1467] Processing CreateTablet for tablet 66ad908770a943af89c4cb090b913670 (DEFAULT_TABLE table=test-table [id=3a6bec095fc14c7293acd1539c13abe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:04.784343 9453 tablet_service.cc:1467] Processing CreateTablet for tablet 66ad908770a943af89c4cb090b913670 (DEFAULT_TABLE table=test-table [id=3a6bec095fc14c7293acd1539c13abe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:04.784433 9453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66ad908770a943af89c4cb090b913670. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:04.784435 9173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66ad908770a943af89c4cb090b913670. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:04.785230 9632 tablet_bootstrap.cc:492] T 66ad908770a943af89c4cb090b913670 P ce649b409ece4d3483a8c8920e127e61: Bootstrap starting.
I20260926 14:56:04.785243 9313 tablet_service.cc:1467] Processing CreateTablet for tablet 66ad908770a943af89c4cb090b913670 (DEFAULT_TABLE table=test-table [id=3a6bec095fc14c7293acd1539c13abe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:04.785339 9313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66ad908770a943af89c4cb090b913670. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:04.785501 9633 tablet_bootstrap.cc:492] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17: Bootstrap starting.
I20260926 14:56:04.785521 9632 tablet_bootstrap.cc:654] T 66ad908770a943af89c4cb090b913670 P ce649b409ece4d3483a8c8920e127e61: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:04.785765 9633 tablet_bootstrap.cc:654] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:04.786273 9633 tablet_bootstrap.cc:492] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17: No bootstrap required, opened a new log
I20260926 14:56:04.786320 9633 ts_tablet_manager.cc:1397] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:04.786417 9633 raft_consensus.cc:348] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:04.786448 9631 tablet_bootstrap.cc:492] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481: Bootstrap starting.
I20260926 14:56:04.786479 9633 raft_consensus.cc:374] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:04.786499 9633 raft_consensus.cc:729] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b060492dd71a43b987c33636317ecd17, State: Initialized, Role: FOLLOWER
I20260926 14:56:04.786535 9633 consensus_queue.cc:260] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:04.786626 9633 ts_tablet_manager.cc:1428] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:04.786808 9631 tablet_bootstrap.cc:654] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:04.786851 9632 tablet_bootstrap.cc:492] T 66ad908770a943af89c4cb090b913670 P ce649b409ece4d3483a8c8920e127e61: No bootstrap required, opened a new log
I20260926 14:56:04.786890 9632 ts_tablet_manager.cc:1397] T 66ad908770a943af89c4cb090b913670 P ce649b409ece4d3483a8c8920e127e61: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260926 14:56:04.786985 9632 raft_consensus.cc:348] T 66ad908770a943af89c4cb090b913670 P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:04.787040 9632 raft_consensus.cc:374] T 66ad908770a943af89c4cb090b913670 P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:04.787056 9632 raft_consensus.cc:729] T 66ad908770a943af89c4cb090b913670 P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce649b409ece4d3483a8c8920e127e61, State: Initialized, Role: FOLLOWER
I20260926 14:56:04.787096 9632 consensus_queue.cc:260] T 66ad908770a943af89c4cb090b913670 P ce649b409ece4d3483a8c8920e127e61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:04.787185 9632 ts_tablet_manager.cc:1428] T 66ad908770a943af89c4cb090b913670 P ce649b409ece4d3483a8c8920e127e61: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:04.787627 9631 tablet_bootstrap.cc:492] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481: No bootstrap required, opened a new log
I20260926 14:56:04.787671 9631 ts_tablet_manager.cc:1397] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:04.787833 9631 raft_consensus.cc:348] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:04.787897 9631 raft_consensus.cc:374] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:04.787917 9631 raft_consensus.cc:729] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4af784ae2f34378ab2dafec3ae54481, State: Initialized, Role: FOLLOWER
I20260926 14:56:04.787954 9631 consensus_queue.cc:260] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:04.788056 9631 ts_tablet_manager.cc:1428] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:04.870383 9642 raft_consensus.cc:482] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:56:04.870481 9642 raft_consensus.cc:504] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:04.870651 9642 leader_election.cc:288] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce649b409ece4d3483a8c8920e127e61 (127.7.37.193:37455), b060492dd71a43b987c33636317ecd17 (127.7.37.195:42517)
I20260926 14:56:04.870821 9193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66ad908770a943af89c4cb090b913670" candidate_uuid: "b4af784ae2f34378ab2dafec3ae54481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce649b409ece4d3483a8c8920e127e61" is_pre_election: true
I20260926 14:56:04.870848 9472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66ad908770a943af89c4cb090b913670" candidate_uuid: "b4af784ae2f34378ab2dafec3ae54481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b060492dd71a43b987c33636317ecd17" is_pre_election: true
I20260926 14:56:04.870932 9472 raft_consensus.cc:2454] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4af784ae2f34378ab2dafec3ae54481 in term 0.
I20260926 14:56:04.870934 9193 raft_consensus.cc:2454] T 66ad908770a943af89c4cb090b913670 P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4af784ae2f34378ab2dafec3ae54481 in term 0.
I20260926 14:56:04.871094 9264 leader_election.cc:302] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4af784ae2f34378ab2dafec3ae54481, ce649b409ece4d3483a8c8920e127e61; no voters:
I20260926 14:56:04.871214 9642 raft_consensus.cc:2781] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:56:04.871250 9642 raft_consensus.cc:482] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:56:04.871272 9642 raft_consensus.cc:3037] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:04.872196 9642 raft_consensus.cc:504] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:04.872331 9642 leader_election.cc:288] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [CANDIDATE]: Term 1 election: Requested vote from peers ce649b409ece4d3483a8c8920e127e61 (127.7.37.193:37455), b060492dd71a43b987c33636317ecd17 (127.7.37.195:42517)
I20260926 14:56:04.872494 9472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66ad908770a943af89c4cb090b913670" candidate_uuid: "b4af784ae2f34378ab2dafec3ae54481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b060492dd71a43b987c33636317ecd17"
I20260926 14:56:04.872488 9193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66ad908770a943af89c4cb090b913670" candidate_uuid: "b4af784ae2f34378ab2dafec3ae54481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce649b409ece4d3483a8c8920e127e61"
I20260926 14:56:04.872563 9472 raft_consensus.cc:3037] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:04.872565 9193 raft_consensus.cc:3037] T 66ad908770a943af89c4cb090b913670 P ce649b409ece4d3483a8c8920e127e61 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:04.873243 9472 raft_consensus.cc:2454] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4af784ae2f34378ab2dafec3ae54481 in term 1.
I20260926 14:56:04.873243 9193 raft_consensus.cc:2454] T 66ad908770a943af89c4cb090b913670 P ce649b409ece4d3483a8c8920e127e61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4af784ae2f34378ab2dafec3ae54481 in term 1.
I20260926 14:56:04.873399 9264 leader_election.cc:302] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4af784ae2f34378ab2dafec3ae54481, ce649b409ece4d3483a8c8920e127e61; no voters:
I20260926 14:56:04.873495 9642 raft_consensus.cc:2781] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:04.873539 9642 raft_consensus.cc:686] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [term 1 LEADER]: Becoming Leader. State: Replica: b4af784ae2f34378ab2dafec3ae54481, State: Running, Role: LEADER
I20260926 14:56:04.873584 9642 consensus_queue.cc:237] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } }
I20260926 14:56:04.874071 9035 catalog_manager.cc:5495] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4af784ae2f34378ab2dafec3ae54481 (127.7.37.194). New cstate: current_term: 1 leader_uuid: "b4af784ae2f34378ab2dafec3ae54481" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4af784ae2f34378ab2dafec3ae54481" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 44771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 } health_report { overall_health: UNKNOWN } } }
I20260926 14:56:04.880616 9193 raft_consensus.cc:1261] T 66ad908770a943af89c4cb090b913670 P ce649b409ece4d3483a8c8920e127e61 [term 1 FOLLOWER]: Refusing update from remote peer b4af784ae2f34378ab2dafec3ae54481: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:04.880627 9472 raft_consensus.cc:1261] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17 [term 1 FOLLOWER]: Refusing update from remote peer b4af784ae2f34378ab2dafec3ae54481: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:04.880805 9642 consensus_queue.cc:1035] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce649b409ece4d3483a8c8920e127e61" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:04.880887 9636 consensus_queue.cc:1035] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b060492dd71a43b987c33636317ecd17" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 14:56:04.884919 9603 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 14:56:06.400075 9036 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:06.400164 9036 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:06.400806 9036 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e83be67a509c459ca1e017fc811a2f3a: Sending DeleteTablet for 3 replicas of tablet 66ad908770a943af89c4cb090b913670
I20260926 14:56:06.401013 9173 tablet_service.cc:1514] Processing DeleteTablet for tablet 66ad908770a943af89c4cb090b913670 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:06 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:59074
I20260926 14:56:06.401031 9313 tablet_service.cc:1514] Processing DeleteTablet for tablet 66ad908770a943af89c4cb090b913670 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:06 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:52466
I20260926 14:56:06.401047 9453 tablet_service.cc:1514] Processing DeleteTablet for tablet 66ad908770a943af89c4cb090b913670 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:06 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:53488
I20260926 14:56:06.401163 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 9108
I20260926 14:56:06.401198 9682 tablet_replica.cc:330] T 66ad908770a943af89c4cb090b913670 P ce649b409ece4d3483a8c8920e127e61: stopping tablet replica
I20260926 14:56:06.402434 9683 tablet_replica.cc:330] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17: stopping tablet replica
I20260926 14:56:06.402487 9683 raft_consensus.cc:2229] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:06.402521 9683 raft_consensus.cc:2258] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:06.402676 9684 tablet_replica.cc:330] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481: stopping tablet replica
I20260926 14:56:06.402768 9684 raft_consensus.cc:2229] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:56:06.402796 9683 ts_tablet_manager.cc:1905] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:06.402833 9684 raft_consensus.cc:2258] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:06.403016 9684 ts_tablet_manager.cc:1905] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:06.404472 9684 ts_tablet_manager.cc:1918] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 14:56:06.404525 9683 ts_tablet_manager.cc:1918] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 14:56:06.404541 9684 log.cc:1196] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/66ad908770a943af89c4cb090b913670
I20260926 14:56:06.404564 9683 log.cc:1196] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/66ad908770a943af89c4cb090b913670
I20260926 14:56:06.404788 9683 ts_tablet_manager.cc:1939] T 66ad908770a943af89c4cb090b913670 P b060492dd71a43b987c33636317ecd17: Deleting consensus metadata
I20260926 14:56:06.404786 9684 ts_tablet_manager.cc:1939] T 66ad908770a943af89c4cb090b913670 P b4af784ae2f34378ab2dafec3ae54481: Deleting consensus metadata
I20260926 14:56:06.405048 9021 catalog_manager.cc:4841] TS b060492dd71a43b987c33636317ecd17 (127.7.37.195:42517): tablet 66ad908770a943af89c4cb090b913670 (table test-table [id=3a6bec095fc14c7293acd1539c13abe8]) successfully deleted
I20260926 14:56:06.405068 9020 catalog_manager.cc:4841] TS b4af784ae2f34378ab2dafec3ae54481 (127.7.37.194:44771): tablet 66ad908770a943af89c4cb090b913670 (table test-table [id=3a6bec095fc14c7293acd1539c13abe8]) successfully deleted
W20260926 14:56:06.406296 9020 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.37.193:37455 (error 108)
I20260926 14:56:06.406490 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 9248
I20260926 14:56:06.407142 9686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:06.406598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:37455 (local address 127.0.0.1:59108)
0926 14:56:06.407077 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:06.407114 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.193:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":51,"threads_started":1}
W20260926 14:56:06.407243 9020 catalog_manager.cc:4584] TS ce649b409ece4d3483a8c8920e127e61 (127.7.37.193:37455): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 66ad908770a943af89c4cb090b913670: Network error: Client connection negotiation failed: client connection to 127.7.37.193:37455: connect: Connection refused (error 111)
I20260926 14:56:06.411298 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 9388
I20260926 14:56:06.415571 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 9004
2026-09-26T14:56:06Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5605 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:56:06 dist-test-slave-f9xb krb5kdc[9689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:56:06 dist-test-slave-f9xb krb5kdc[9689](info): set up 2 sockets
Sep 26 14:56:06 dist-test-slave-f9xb krb5kdc[9689](info): commencing operation
krb5kdc: starting...
W20260926 14:56:07.355038 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.920s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:56:07 dist-test-slave-f9xb krb5kdc[9689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:56:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:56:07.454814 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:43469
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:43469
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:07.528679 9702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:07.528847 9702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:07.528867 9702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:07.529944 9702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:07.529982 9702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:07.529995 9702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:07.530005 9702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:07.530017 9702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:56:07.531414 9702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:43469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:43469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:07.531616 9702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:07.531818 9702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:07.536029 9710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:07.536052 9707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:07.535990 9708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:07.536136 9702 server_base.cc:1034] running on GCE node
I20260926 14:56:07.536348 9702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:07.536594 9702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:07.537720 9702 hybrid_clock.cc:648] HybridClock initialized: now 1790434567537713 us; error 28 us; skew 500 ppm
Sep 26 14:56:07 dist-test-slave-f9xb krb5kdc[9689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434567, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:07.539860 9702 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:56:07.540236 9702 webserver.cc:460] Webserver started at http://127.7.37.254:39019/ using document root <none> and password file <none>
I20260926 14:56:07.540409 9702 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:07.540450 9702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:07.540571 9702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:07.541445 9702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "42cc8d59ac0940a6916bc4c8e07fb550"
format_stamp: "Formatted at 2026-09-26 14:56:07 on dist-test-slave-f9xb"
server_key: "53755e49464fabe8eaffa65243bc1f7c"
server_key_iv: "3acb10c2e72651f1f0d5424aba079c67"
server_key_version: "encryptionkey@0"
I20260926 14:56:07.541759 9702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "42cc8d59ac0940a6916bc4c8e07fb550"
format_stamp: "Formatted at 2026-09-26 14:56:07 on dist-test-slave-f9xb"
server_key: "53755e49464fabe8eaffa65243bc1f7c"
server_key_iv: "3acb10c2e72651f1f0d5424aba079c67"
server_key_version: "encryptionkey@0"
I20260926 14:56:07.542838 9702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:07.543428 9717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:07.543576 9702 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:07.543648 9702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "42cc8d59ac0940a6916bc4c8e07fb550"
format_stamp: "Formatted at 2026-09-26 14:56:07 on dist-test-slave-f9xb"
server_key: "53755e49464fabe8eaffa65243bc1f7c"
server_key_iv: "3acb10c2e72651f1f0d5424aba079c67"
server_key_version: "encryptionkey@0"
I20260926 14:56:07.543709 9702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:07.550714 9702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:07.552590 9702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:07.552706 9702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:07.556075 9702 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:43469
I20260926 14:56:07.556103 9769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:43469 every 8 connection(s)
I20260926 14:56:07.556420 9702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:56:07.557215 9770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:07.557993 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 9702
I20260926 14:56:07.558069 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:56:07.558221 7319 external_mini_cluster.cc:1428] Setting key 795f74636c6581c2c0d58c7869963556
I20260926 14:56:07.559410 9770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550: Bootstrap starting.
I20260926 14:56:07.560068 9770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:07.560338 9770 log.cc:824] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:07.560964 9770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550: No bootstrap required, opened a new log
I20260926 14:56:07.563010 9770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42cc8d59ac0940a6916bc4c8e07fb550" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43469 } }
I20260926 14:56:07.563149 9770 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:07.563185 9770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42cc8d59ac0940a6916bc4c8e07fb550, State: Initialized, Role: FOLLOWER
I20260926 14:56:07.563304 9770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42cc8d59ac0940a6916bc4c8e07fb550" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43469 } }
I20260926 14:56:07.563380 9770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:56:07.563408 9770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:56:07.563458 9770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:07.564093 9770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42cc8d59ac0940a6916bc4c8e07fb550" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43469 } }
I20260926 14:56:07.564191 9770 leader_election.cc:302] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42cc8d59ac0940a6916bc4c8e07fb550; no voters:
I20260926 14:56:07.564337 9770 leader_election.cc:288] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:56:07.564384 9775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:07.564553 9770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:07.564597 9775 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550 [term 1 LEADER]: Becoming Leader. State: Replica: 42cc8d59ac0940a6916bc4c8e07fb550, State: Running, Role: LEADER
Sep 26 14:56:07 dist-test-slave-f9xb krb5kdc[9689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:07.564718 9775 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42cc8d59ac0940a6916bc4c8e07fb550" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43469 } }
I20260926 14:56:07.565045 9777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42cc8d59ac0940a6916bc4c8e07fb550. Latest consensus state: current_term: 1 leader_uuid: "42cc8d59ac0940a6916bc4c8e07fb550" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42cc8d59ac0940a6916bc4c8e07fb550" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43469 } } }
I20260926 14:56:07.565121 9777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:07.565291 9776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42cc8d59ac0940a6916bc4c8e07fb550" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42cc8d59ac0940a6916bc4c8e07fb550" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43469 } } }
I20260926 14:56:07.565657 9776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550 [sys.catalog]: This master's current role is: LEADER
W20260926 14:56:07.566251 9791 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 14:56:07.567224 9791 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 14:56:07.567287 9784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:56:07.567440 9784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:56:07.568758 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:07.558642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53536 (local address 127.7.37.254:43469)
0926 14:56:07.558787 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:07.558795 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:07.558820 (+ 25us) server_negotiation.cc:407] Connection header received
0926 14:56:07.559250 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:07.559256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:07.559297 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:07.559399 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:07.560185 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:07.561230 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:07.562274 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:07.562655 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:07.562735 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:07.565253 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:07.565258 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:07.565258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:07.565268 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:07.567047 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:07.567444 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:07.567448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:07.567448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:07.567567 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:07.567792 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:07.567793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:07.567793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:07.568053 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:07.568187 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:07.568435 (+ 248us) server_negotiation.cc:299] Negotiation successful
0926 14:56:07.568558 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260926 14:56:07.571578 9784 catalog_manager.cc:1347] Generated new cluster ID: d49739e0f45648a9ab758e434f65158b
I20260926 14:56:07.571621 9784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:56:07.585822 9784 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:56:07.586477 9784 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:07.596230 9784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42cc8d59ac0940a6916bc4c8e07fb550: Generated new TSK 0
I20260926 14:56:07.596367 9784 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
I20260926 14:56:07.622615 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:43469
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:07.696532 9796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:07.696700 9796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:07.696720 9796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:07.697813 9796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:07.697851 9796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:07.697877 9796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:56:07.699210 9796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:43469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:07.699394 9796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:07.699579 9796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:07.703913 9802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:07.703935 9804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:07.704037 9796 server_base.cc:1034] running on GCE node
W20260926 14:56:07.703913 9801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:07.704308 9796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:07.704548 9796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:07.705681 9796 hybrid_clock.cc:648] HybridClock initialized: now 1790434567705665 us; error 26 us; skew 500 ppm
Sep 26 14:56:07 dist-test-slave-f9xb krb5kdc[9689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434567, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:07.707966 9796 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:56:07.708314 9796 webserver.cc:460] Webserver started at http://127.7.37.193:45801/ using document root <none> and password file <none>
I20260926 14:56:07.708500 9796 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:07.708546 9796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:07.708657 9796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:07.709554 9796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "2b050653140745a3bc4605516ffc49bb"
format_stamp: "Formatted at 2026-09-26 14:56:07 on dist-test-slave-f9xb"
server_key: "12ec80fb96f869bd0beca35870947013"
server_key_iv: "2e98e8273aeafd4c09678dfe154bc363"
server_key_version: "encryptionkey@0"
I20260926 14:56:07.709843 9796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "2b050653140745a3bc4605516ffc49bb"
format_stamp: "Formatted at 2026-09-26 14:56:07 on dist-test-slave-f9xb"
server_key: "12ec80fb96f869bd0beca35870947013"
server_key_iv: "2e98e8273aeafd4c09678dfe154bc363"
server_key_version: "encryptionkey@0"
I20260926 14:56:07.710959 9796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:07.711544 9811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:07.711694 9796 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:07.711782 9796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "2b050653140745a3bc4605516ffc49bb"
format_stamp: "Formatted at 2026-09-26 14:56:07 on dist-test-slave-f9xb"
server_key: "12ec80fb96f869bd0beca35870947013"
server_key_iv: "2e98e8273aeafd4c09678dfe154bc363"
server_key_version: "encryptionkey@0"
I20260926 14:56:07.711839 9796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:07.733881 9796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:07.735494 9796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:07.735589 9796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:07.735730 9796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:07.736011 9796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:07.736042 9796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:07.736060 9796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:07.736070 9796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:07.741079 9796 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:38423
I20260926 14:56:07.741156 9924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:38423 every 8 connection(s)
I20260926 14:56:07.741365 9796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:56:07.746057 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 9796
I20260926 14:56:07.746156 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:56:07.746327 7319 external_mini_cluster.cc:1428] Setting key 38c6aad1bcd2439721c689725abe5a39
Sep 26 14:56:07 dist-test-slave-f9xb krb5kdc[9689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434567, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:07.749488 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:07.741933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:36981 (local address 127.7.37.254:43469)
0926 14:56:07.741975 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:07.741977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:07.742151 (+ 174us) server_negotiation.cc:407] Connection header received
0926 14:56:07.742552 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:07.742553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:07.742561 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:07.742611 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:07.743379 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:07.743914 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:07.744806 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:07.744993 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:07.745032 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:07.746981 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:07.746982 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:07.746982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:07.746989 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:07.748339 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:07.748685 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:07.748686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:07.748687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:07.748738 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:07.748939 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:07.748940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:07.748940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:07.749039 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:07.749099 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:07.749328 (+ 229us) server_negotiation.cc:299] Negotiation successful
0926 14:56:07.749401 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 14:56:07.749643 9927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:07.741904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43469 (local address 127.7.37.193:36981)
0926 14:56:07.742045 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:07.742080 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:07.742469 (+ 389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:07.742663 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:07.742666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:07.743153 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:07.743318 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:07.743323 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:07.743981 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:07.743981 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:07.744743 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:07.744745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:07.745041 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:07.745042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:07.745127 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:07.745467 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:07.745474 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:07.746873 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:07.748411 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:07.748416 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:07.748418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:07.748621 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:07.748788 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:07.748789 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:07.748789 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:07.748887 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:07.749091 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:07.749097 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:07.749221 (+ 124us) client_negotiation.cc:769] Sending connection context
0926 14:56:07.749304 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 14:56:07.749441 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260926 14:56:07.750025 9925 heartbeater.cc:347] Connected to a master server at 127.7.37.254:43469
I20260926 14:56:07.750114 9925 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:07.750277 9925 heartbeater.cc:511] Master 127.7.37.254:43469 requested a full tablet report, sending...
I20260926 14:56:07.750734 9734 ts_manager.cc:194] Registered new tserver with Master: 2b050653140745a3bc4605516ffc49bb (127.7.37.193:38423)
I20260926 14:56:07.751313 9734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:36981
WARNING: no policy specified for kudu/127.7.37.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
I20260926 14:56:07.766654 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:43469
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:07.842056 9930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:07.842226 9930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:07.842249 9930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:07.843317 9930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:07.843356 9930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:07.843382 9930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:56:07.844810 9930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:43469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:07.844977 9930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:07.845137 9930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:07.849529 9935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:07.849617 9930 server_base.cc:1034] running on GCE node
W20260926 14:56:07.849567 9938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:07.849536 9936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:07.849907 9930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:07.850109 9930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:07.851241 9930 hybrid_clock.cc:648] HybridClock initialized: now 1790434567851228 us; error 24 us; skew 500 ppm
Sep 26 14:56:07 dist-test-slave-f9xb krb5kdc[9689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434567, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:07.853526 9930 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:56:07.853916 9930 webserver.cc:460] Webserver started at http://127.7.37.194:44327/ using document root <none> and password file <none>
I20260926 14:56:07.854097 9930 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:07.854142 9930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:07.854255 9930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:07.855131 9930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "de0b1d8155294d4dacfb30c12a7c7a73"
format_stamp: "Formatted at 2026-09-26 14:56:07 on dist-test-slave-f9xb"
server_key: "ec81c04ad4bb8eb5874c8f23197f443e"
server_key_iv: "38df24fd43d80af9dd4368b3fd1fe354"
server_key_version: "encryptionkey@0"
I20260926 14:56:07.855422 9930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "de0b1d8155294d4dacfb30c12a7c7a73"
format_stamp: "Formatted at 2026-09-26 14:56:07 on dist-test-slave-f9xb"
server_key: "ec81c04ad4bb8eb5874c8f23197f443e"
server_key_iv: "38df24fd43d80af9dd4368b3fd1fe354"
server_key_version: "encryptionkey@0"
I20260926 14:56:07.856575 9930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 14:56:07.857188 9945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:07.857332 9930 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:07.857403 9930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "de0b1d8155294d4dacfb30c12a7c7a73"
format_stamp: "Formatted at 2026-09-26 14:56:07 on dist-test-slave-f9xb"
server_key: "ec81c04ad4bb8eb5874c8f23197f443e"
server_key_iv: "38df24fd43d80af9dd4368b3fd1fe354"
server_key_version: "encryptionkey@0"
I20260926 14:56:07.857457 9930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:07.872361 9930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:07.873956 9930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:07.874075 9930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:07.874254 9930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:07.874478 9930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:07.874507 9930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:07.874532 9930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:07.874549 9930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:07.879649 9930 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:44981
I20260926 14:56:07.879688 10058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:44981 every 8 connection(s)
I20260926 14:56:07.879936 9930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:56:07.880211 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 9930
I20260926 14:56:07.880298 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:56:07.880447 7319 external_mini_cluster.cc:1428] Setting key c6abea60fe91a49fad66a50933556e14
Sep 26 14:56:07 dist-test-slave-f9xb krb5kdc[9689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434567, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.37.195@KRBTEST.COM; defaulting to no policy
I20260926 14:56:07.889379 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:07.880560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:48311 (local address 127.7.37.254:43469)
0926 14:56:07.880734 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:07.880736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:07.880969 (+ 233us) server_negotiation.cc:407] Connection header received
0926 14:56:07.881268 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:07.881269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:07.881279 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:07.881321 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:07.882343 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:07.883093 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:07.884499 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:07.884721 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:07.884783 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:07.886804 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:07.886806 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:07.886806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:07.886814 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:07.888158 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:07.888519 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:07.888520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:07.888521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:07.888574 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:07.888786 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:07.888787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:07.888788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:07.888910 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:07.888968 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:07.889211 (+ 243us) server_negotiation.cc:299] Negotiation successful
0926 14:56:07.889288 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260926 14:56:07.889505 10061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:07.880530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43469 (local address 127.7.37.194:48311)
0926 14:56:07.880670 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:07.880704 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:07.881170 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:07.881380 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:07.881383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:07.882023 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:07.882254 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:07.882260 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:07.883178 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:07.883180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:07.884410 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:07.884412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:07.884801 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:07.884802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:07.884931 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:07.885324 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:07.885333 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:07.886715 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:07.888224 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:07.888231 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:07.888233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:07.888443 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:07.888627 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:07.888628 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:07.888628 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:07.888728 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:07.888967 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:07.888972 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:07.889095 (+ 123us) client_negotiation.cc:769] Sending connection context
0926 14:56:07.889182 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 14:56:07.889316 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":81,"threads_started":1}
I20260926 14:56:07.889901 10059 heartbeater.cc:347] Connected to a master server at 127.7.37.254:43469
I20260926 14:56:07.889986 10059 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:07.890136 10059 heartbeater.cc:511] Master 127.7.37.254:43469 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.195@KRBTEST.COM" created.
I20260926 14:56:07.890471 9734 ts_manager.cc:194] Registered new tserver with Master: de0b1d8155294d4dacfb30c12a7c7a73 (127.7.37.194:44981)
I20260926 14:56:07.890882 9734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:48311
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
I20260926 14:56:07.902833 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:43469
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:07.976831 10064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:07.977003 10064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:07.977023 10064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:07.978121 10064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:07.978158 10064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:07.978183 10064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:56:07.979511 10064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:43469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:07.979687 10064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:07.979902 10064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:07.984158 10072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:07.984269 10064 server_base.cc:1034] running on GCE node
W20260926 14:56:07.984243 10069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:07.984185 10070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:07.984541 10064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:07.984747 10064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:07.985882 10064 hybrid_clock.cc:648] HybridClock initialized: now 1790434567985869 us; error 26 us; skew 500 ppm
Sep 26 14:56:07 dist-test-slave-f9xb krb5kdc[9689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434567, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:07.988024 10064 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:56:07.988404 10064 webserver.cc:460] Webserver started at http://127.7.37.195:44825/ using document root <none> and password file <none>
I20260926 14:56:07.988595 10064 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:07.988639 10064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:07.988749 10064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:07.989616 10064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "e80811efe995449da27f6b831d361451"
format_stamp: "Formatted at 2026-09-26 14:56:07 on dist-test-slave-f9xb"
server_key: "103d8bf1acd29ab9ad3de59f33062044"
server_key_iv: "53da166bac6db25a381d927b01b6181c"
server_key_version: "encryptionkey@0"
I20260926 14:56:07.989909 10064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "e80811efe995449da27f6b831d361451"
format_stamp: "Formatted at 2026-09-26 14:56:07 on dist-test-slave-f9xb"
server_key: "103d8bf1acd29ab9ad3de59f33062044"
server_key_iv: "53da166bac6db25a381d927b01b6181c"
server_key_version: "encryptionkey@0"
I20260926 14:56:07.991004 10064 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:07.991659 10079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:07.991824 10064 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:07.991915 10064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "e80811efe995449da27f6b831d361451"
format_stamp: "Formatted at 2026-09-26 14:56:07 on dist-test-slave-f9xb"
server_key: "103d8bf1acd29ab9ad3de59f33062044"
server_key_iv: "53da166bac6db25a381d927b01b6181c"
server_key_version: "encryptionkey@0"
I20260926 14:56:07.991971 10064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:08.004803 10064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:08.006377 10064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:08.006491 10064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:08.006655 10064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:08.006899 10064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:08.006930 10064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:08.006954 10064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:08.006978 10064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:08.012205 10064 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:41761
I20260926 14:56:08.012254 10192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:41761 every 8 connection(s)
I20260926 14:56:08.012497 10064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:56:08.016327 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 10064
I20260926 14:56:08.016461 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:56:08.016646 7319 external_mini_cluster.cc:1428] Setting key 3a17a1db86f8b0938717cfb5192c0a6e
Sep 26 14:56:08 dist-test-slave-f9xb krb5kdc[9689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434567, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:08.020706 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:08.013042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:48753 (local address 127.7.37.254:43469)
0926 14:56:08.013081 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:08.013083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:08.013251 (+ 168us) server_negotiation.cc:407] Connection header received
0926 14:56:08.013686 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:08.013688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:08.013698 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:08.013749 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:08.014633 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:08.015119 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:08.016133 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:08.016341 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:08.016394 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:08.018444 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:08.018446 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:08.018446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:08.018453 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:08.019516 (+ 1063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:08.019897 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:08.019898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:08.019899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:08.019951 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:08.020154 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:08.020155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:08.020155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:08.020255 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:08.020309 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:08.020545 (+ 236us) server_negotiation.cc:299] Negotiation successful
0926 14:56:08.020619 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 14:56:08.020830 10195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:08.012999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43469 (local address 127.7.37.195:48753)
0926 14:56:08.013150 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:08.013180 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:08.013599 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:08.013809 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:08.013812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:08.014370 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:08.014547 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:08.014552 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:08.015194 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:08.015195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:08.016059 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:08.016061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:08.016404 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:08.016405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:08.016522 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:08.017003 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:08.017012 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:08.018357 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:08.019584 (+ 1227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:08.019590 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:08.019592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:08.019836 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:08.020003 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:08.020004 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:08.020004 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:08.020102 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:08.020310 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:08.020315 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:08.020434 (+ 119us) client_negotiation.cc:769] Sending connection context
0926 14:56:08.020516 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 14:56:08.020646 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
I20260926 14:56:08.021193 10193 heartbeater.cc:347] Connected to a master server at 127.7.37.254:43469
I20260926 14:56:08.021271 10193 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:08.021441 10193 heartbeater.cc:511] Master 127.7.37.254:43469 requested a full tablet report, sending...
I20260926 14:56:08.021749 9734 ts_manager.cc:194] Registered new tserver with Master: e80811efe995449da27f6b831d361451 (127.7.37.195:41761)
I20260926 14:56:08.022075 9734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:48753
I20260926 14:56:08.028206 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 14:56:08.037413 9773 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:08.036696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53548 (local address 127.7.37.254:43469)
0926 14:56:08.036744 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:08.036746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:08.036819 (+ 73us) server_negotiation.cc:407] Connection header received
0926 14:56:08.036877 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:08.036877 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:08.036887 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:08.036929 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:08.037362 (+ 433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53548: BlockingRecv error: recv got EOF from 127.0.0.1:53548 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 14:56:08.038203 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 9796
I20260926 14:56:08.041486 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 9930
I20260926 14:56:08.044847 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 10064
I20260926 14:56:08.048252 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 9702
2026-09-26T14:56:08Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1631 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:56:08 dist-test-slave-f9xb krb5kdc[10205](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:56:08 dist-test-slave-f9xb krb5kdc[10205](info): set up 2 sockets
Sep 26 14:56:08 dist-test-slave-f9xb krb5kdc[10205](info): commencing operation
krb5kdc: starting...
W20260926 14:56:08.987210 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.921s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:56:09 dist-test-slave-f9xb krb5kdc[10205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:56:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:56:09.092296 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:41317
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:45213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:41317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:09.167140 10218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:09.167320 10218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:09.167338 10218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:09.168443 10218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:09.168484 10218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:09.168496 10218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:09.168506 10218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:09.168517 10218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:56:09.169850 10218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:45213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:41317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:41317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:09.170015 10218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:09.170166 10218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:09.174415 10223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:09.174515 10224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:09.174568 10218 server_base.cc:1034] running on GCE node
W20260926 14:56:09.174448 10226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:09.174800 10218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:09.175037 10218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:09.176162 10218 hybrid_clock.cc:648] HybridClock initialized: now 1790434569176144 us; error 39 us; skew 500 ppm
Sep 26 14:56:09 dist-test-slave-f9xb krb5kdc[10205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434569, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:09.178391 10218 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:56:09.178785 10218 webserver.cc:460] Webserver started at http://127.7.37.254:45733/ using document root <none> and password file <none>
I20260926 14:56:09.178958 10218 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:09.179003 10218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:09.179113 10218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:09.180078 10218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "5c3944d5aeb745eb8727c2c9c9e33121"
format_stamp: "Formatted at 2026-09-26 14:56:09 on dist-test-slave-f9xb"
server_key: "0685044792a292480120d548f764cff3"
server_key_iv: "e8c179176d47038ecd996a36cd16a730"
server_key_version: "encryptionkey@0"
I20260926 14:56:09.180371 10218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "5c3944d5aeb745eb8727c2c9c9e33121"
format_stamp: "Formatted at 2026-09-26 14:56:09 on dist-test-slave-f9xb"
server_key: "0685044792a292480120d548f764cff3"
server_key_iv: "e8c179176d47038ecd996a36cd16a730"
server_key_version: "encryptionkey@0"
I20260926 14:56:09.181491 10218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:09.182164 10233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:09.182307 10218 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:09.182379 10218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "5c3944d5aeb745eb8727c2c9c9e33121"
format_stamp: "Formatted at 2026-09-26 14:56:09 on dist-test-slave-f9xb"
server_key: "0685044792a292480120d548f764cff3"
server_key_iv: "e8c179176d47038ecd996a36cd16a730"
server_key_version: "encryptionkey@0"
I20260926 14:56:09.182432 10218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:09.195048 10218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:09.196933 10218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:09.197053 10218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:09.200495 10218 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:41317
I20260926 14:56:09.200523 10285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:41317 every 8 connection(s)
I20260926 14:56:09.200801 10218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:56:09.201527 10286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:09.203579 10286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121: Bootstrap starting.
I20260926 14:56:09.204128 10286 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:09.204370 10286 log.cc:824] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:09.204977 10286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121: No bootstrap required, opened a new log
I20260926 14:56:09.205931 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 10218
I20260926 14:56:09.206012 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:56:09.206156 7319 external_mini_cluster.cc:1428] Setting key 2caf2e6db888b8622b0aff62dd4ee5d9
I20260926 14:56:09.206645 10286 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3944d5aeb745eb8727c2c9c9e33121" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 41317 } }
I20260926 14:56:09.206800 10286 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:09.206825 10286 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3944d5aeb745eb8727c2c9c9e33121, State: Initialized, Role: FOLLOWER
I20260926 14:56:09.206943 10286 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3944d5aeb745eb8727c2c9c9e33121" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 41317 } }
I20260926 14:56:09.207031 10286 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:56:09.207067 10286 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:56:09.207109 10286 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:09.207656 10286 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3944d5aeb745eb8727c2c9c9e33121" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 41317 } }
I20260926 14:56:09.207775 10286 leader_election.cc:302] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c3944d5aeb745eb8727c2c9c9e33121; no voters:
I20260926 14:56:09.207932 10286 leader_election.cc:288] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:56:09.207973 10291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:09.208112 10291 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3944d5aeb745eb8727c2c9c9e33121, State: Running, Role: LEADER
I20260926 14:56:09.208168 10286 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:09.208213 10291 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3944d5aeb745eb8727c2c9c9e33121" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 41317 } }
I20260926 14:56:09.208549 10291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c3944d5aeb745eb8727c2c9c9e33121. Latest consensus state: current_term: 1 leader_uuid: "5c3944d5aeb745eb8727c2c9c9e33121" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3944d5aeb745eb8727c2c9c9e33121" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 41317 } } }
I20260926 14:56:09.208559 10293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c3944d5aeb745eb8727c2c9c9e33121" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3944d5aeb745eb8727c2c9c9e33121" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 41317 } } }
I20260926 14:56:09.208657 10291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:09.208698 10293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:09.208855 10298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:56:09.209512 10298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:56:09.211053 10298 catalog_manager.cc:1347] Generated new cluster ID: abe14fe6897e46a49a3f16c015f33edf
I20260926 14:56:09.211102 10298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 14:56:09 dist-test-slave-f9xb krb5kdc[10205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:09.217267 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:09.206573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47400 (local address 127.7.37.254:41317)
0926 14:56:09.207014 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:09.207023 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:09.207053 (+ 30us) server_negotiation.cc:407] Connection header received
0926 14:56:09.207172 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:09.207179 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:09.207231 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:09.207364 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:09.207951 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:09.209318 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:09.210294 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:09.210867 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:09.210972 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:09.213502 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:09.213506 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:09.213506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:09.213516 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:09.215688 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:09.216077 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:09.216079 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:09.216080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:09.216171 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:09.216351 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:09.216351 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:09.216351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:09.216612 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:09.216741 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:09.216854 (+ 113us) server_negotiation.cc:299] Negotiation successful
0926 14:56:09.217069 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":363,"threads_started":1}
I20260926 14:56:09.218179 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:56:09.223887 10298 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:56:09.224401 10298 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:09.235553 10298 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c3944d5aeb745eb8727c2c9c9e33121: Generated new TSK 0
I20260926 14:56:09.235708 10298 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:56:09.239224 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:09.235888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47406 (local address 127.7.37.254:41317)
0926 14:56:09.235944 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:09.235946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:09.236116 (+ 170us) server_negotiation.cc:407] Connection header received
0926 14:56:09.236177 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:09.236178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:09.236186 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:09.236220 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:09.236607 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:09.237074 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:09.238088 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:09.238274 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:09.238316 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:09.238884 (+ 568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:09.238885 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:09.238885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:09.239039 (+ 154us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:09.239053 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:09.239058 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:09.239097 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:09.239163 (+ 66us) server_negotiation.cc:299] Negotiation successful
0926 14:56:09.239179 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 14:56:09.241467 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 10218
2026-09-26T14:56:09Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1191 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:56:09 dist-test-slave-f9xb krb5kdc[10319](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:56:09 dist-test-slave-f9xb krb5kdc[10319](info): set up 2 sockets
Sep 26 14:56:09 dist-test-slave-f9xb krb5kdc[10319](info): commencing operation
krb5kdc: starting...
W20260926 14:56:10.183156 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:56:10 dist-test-slave-f9xb krb5kdc[10319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:56:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:56:10.286455 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:46415
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:46415
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:10.364529 10332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:10.364703 10332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:10.364732 10332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:10.365798 10332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:10.365835 10332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:10.365855 10332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:10.365869 10332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:10.365885 10332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:56:10.367257 10332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:46415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:46415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:10.367480 10332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:10.367662 10332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:10.371842 10337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:10.371865 10340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:10.371956 10332 server_base.cc:1034] running on GCE node
W20260926 14:56:10.371882 10338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:10.372278 10332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:10.372655 10332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:10.373790 10332 hybrid_clock.cc:648] HybridClock initialized: now 1790434570373772 us; error 34 us; skew 500 ppm
Sep 26 14:56:10 dist-test-slave-f9xb krb5kdc[10319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434570, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:10.379148 10332 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:56:10.379555 10332 webserver.cc:460] Webserver started at http://127.7.37.254:42755/ using document root <none> and password file <none>
I20260926 14:56:10.379777 10332 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:10.379829 10332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:10.379913 10332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:10.380771 10332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "4e1fd1f53c8944be918f0c6231c4eb40"
format_stamp: "Formatted at 2026-09-26 14:56:10 on dist-test-slave-f9xb"
server_key: "94fdae1c154f6987f850d9e0dce90eee"
server_key_iv: "2ee2a88e8dafb30e3b9978e521879032"
server_key_version: "encryptionkey@0"
I20260926 14:56:10.381035 10332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "4e1fd1f53c8944be918f0c6231c4eb40"
format_stamp: "Formatted at 2026-09-26 14:56:10 on dist-test-slave-f9xb"
server_key: "94fdae1c154f6987f850d9e0dce90eee"
server_key_iv: "2ee2a88e8dafb30e3b9978e521879032"
server_key_version: "encryptionkey@0"
I20260926 14:56:10.382067 10332 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:10.382639 10347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:10.382784 10332 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:10.382858 10332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "4e1fd1f53c8944be918f0c6231c4eb40"
format_stamp: "Formatted at 2026-09-26 14:56:10 on dist-test-slave-f9xb"
server_key: "94fdae1c154f6987f850d9e0dce90eee"
server_key_iv: "2ee2a88e8dafb30e3b9978e521879032"
server_key_version: "encryptionkey@0"
I20260926 14:56:10.382915 10332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:10.395807 10332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:10.397259 10332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:10.397352 10332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:10.400606 10332 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:46415
I20260926 14:56:10.400647 10399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:46415 every 8 connection(s)
I20260926 14:56:10.400909 10332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:56:10.401719 10400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:10.403566 10400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40: Bootstrap starting.
I20260926 14:56:10.404076 10400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:10.404311 10400 log.cc:824] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:10.404876 10400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40: No bootstrap required, opened a new log
I20260926 14:56:10.406294 10400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e1fd1f53c8944be918f0c6231c4eb40" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 46415 } }
I20260926 14:56:10.406411 10400 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:10.406440 10400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e1fd1f53c8944be918f0c6231c4eb40, State: Initialized, Role: FOLLOWER
I20260926 14:56:10.406559 10400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e1fd1f53c8944be918f0c6231c4eb40" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 46415 } }
I20260926 14:56:10.406630 10400 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:56:10.406659 10400 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:56:10.406704 10400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:10.407410 10400 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e1fd1f53c8944be918f0c6231c4eb40" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 46415 } }
I20260926 14:56:10.407492 10400 leader_election.cc:302] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e1fd1f53c8944be918f0c6231c4eb40; no voters:
I20260926 14:56:10.407624 10400 leader_election.cc:288] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:56:10.407661 10403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:10.407842 10403 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40 [term 1 LEADER]: Becoming Leader. State: Replica: 4e1fd1f53c8944be918f0c6231c4eb40, State: Running, Role: LEADER
I20260926 14:56:10.407858 10400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:10.407963 10403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e1fd1f53c8944be918f0c6231c4eb40" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 46415 } }
I20260926 14:56:10.408278 10404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e1fd1f53c8944be918f0c6231c4eb40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e1fd1f53c8944be918f0c6231c4eb40" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 46415 } } }
I20260926 14:56:10.408301 10405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e1fd1f53c8944be918f0c6231c4eb40. Latest consensus state: current_term: 1 leader_uuid: "4e1fd1f53c8944be918f0c6231c4eb40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e1fd1f53c8944be918f0c6231c4eb40" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 46415 } } }
I20260926 14:56:10.408399 10404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:10.408416 10405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:10.408695 10412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:56:10.409150 10412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:56:10.409964 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 10332
I20260926 14:56:10.410068 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:56:10.410238 7319 external_mini_cluster.cc:1428] Setting key bed784363f6543add27af3caf6c324c4
I20260926 14:56:10.410476 10412 catalog_manager.cc:1347] Generated new cluster ID: cd833e6159f0488a838836b4cdd7aeb7
I20260926 14:56:10.410523 10412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 14:56:10 dist-test-slave-f9xb krb5kdc[10319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:10.419461 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:10.410612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59322 (local address 127.7.37.254:46415)
0926 14:56:10.411838 (+ 1226us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:10.411845 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:10.411867 (+ 22us) server_negotiation.cc:407] Connection header received
0926 14:56:10.411961 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:10.411966 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:10.412002 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:10.412110 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:10.412667 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:10.413525 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:10.414220 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:10.414614 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:10.414700 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:10.416670 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:10.416674 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:10.416674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:10.416685 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:10.418075 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:10.418374 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:10.418383 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:10.418384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:10.418483 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:10.418699 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:10.418700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:10.418700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:10.418924 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:10.419055 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:10.419150 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 14:56:10.419280 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1191,"thread_start_us":67,"threads_started":1}
I20260926 14:56:10.439150 10412 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:56:10.439618 10412 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:10.444872 10412 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e1fd1f53c8944be918f0c6231c4eb40: Generated new TSK 0
I20260926 14:56:10.444999 10412 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
I20260926 14:56:10.476888 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:46415
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:10.577080 10426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:10.577248 10426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:10.577270 10426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:10.578310 10426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:10.578339 10426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:10.578363 10426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:56:10.579669 10426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:46415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:10.579862 10426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:10.580065 10426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:10.584439 10431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:10.584452 10434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:10.584470 10426 server_base.cc:1034] running on GCE node
W20260926 14:56:10.584465 10432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:10.584826 10426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:10.585071 10426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:10.586223 10426 hybrid_clock.cc:648] HybridClock initialized: now 1790434570586208 us; error 30 us; skew 500 ppm
Sep 26 14:56:10 dist-test-slave-f9xb krb5kdc[10319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434570, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:10.588521 10426 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:56:10.588864 10426 webserver.cc:460] Webserver started at http://127.7.37.193:38719/ using document root <none> and password file <none>
I20260926 14:56:10.589051 10426 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:10.589095 10426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:10.589210 10426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:10.590106 10426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "b3f080c27bab478dbdf7b7d3e73c206b"
format_stamp: "Formatted at 2026-09-26 14:56:10 on dist-test-slave-f9xb"
server_key: "7b3d602c4f3a9524ca09633c24710a9a"
server_key_iv: "a653e8a2b1419bd01cc07346c48f0776"
server_key_version: "encryptionkey@0"
I20260926 14:56:10.590391 10426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "b3f080c27bab478dbdf7b7d3e73c206b"
format_stamp: "Formatted at 2026-09-26 14:56:10 on dist-test-slave-f9xb"
server_key: "7b3d602c4f3a9524ca09633c24710a9a"
server_key_iv: "a653e8a2b1419bd01cc07346c48f0776"
server_key_version: "encryptionkey@0"
I20260926 14:56:10.591511 10426 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 14:56:10.592162 10441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:10.592299 10426 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:10.592370 10426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "b3f080c27bab478dbdf7b7d3e73c206b"
format_stamp: "Formatted at 2026-09-26 14:56:10 on dist-test-slave-f9xb"
server_key: "7b3d602c4f3a9524ca09633c24710a9a"
server_key_iv: "a653e8a2b1419bd01cc07346c48f0776"
server_key_version: "encryptionkey@0"
I20260926 14:56:10.592430 10426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:10.604583 10426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:10.606101 10426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:10.606215 10426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:10.606375 10426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:10.606606 10426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:10.606635 10426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:10.606659 10426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:10.606683 10426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:10.611811 10426 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:38305
I20260926 14:56:10.611853 10554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:38305 every 8 connection(s)
I20260926 14:56:10.612102 10426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
Sep 26 14:56:10 dist-test-slave-f9xb krb5kdc[10319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434570, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:10.619798 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:10.612707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:42585 (local address 127.7.37.254:46415)
0926 14:56:10.612756 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:10.612758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:10.612952 (+ 194us) server_negotiation.cc:407] Connection header received
0926 14:56:10.613349 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:10.613349 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:10.613359 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:10.613420 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:10.614216 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:10.614687 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:10.615616 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:10.615855 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:10.615899 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:10.617707 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:10.617708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:10.617709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:10.617715 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:10.618784 (+ 1069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:10.619109 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:10.619110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:10.619110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:10.619149 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:10.619310 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:10.619310 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:10.619311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:10.619406 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:10.619449 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:10.619643 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 14:56:10.619700 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 14:56:10.619933 10557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:10.612685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:46415 (local address 127.7.37.193:42585)
0926 14:56:10.612837 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:10.612875 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:10.613281 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:10.613474 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:10.613477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:10.613984 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:10.614149 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:10.614153 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:10.614748 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:10.614749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:10.615544 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:10.615547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:10.615903 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:10.615903 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:10.615992 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:10.616347 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:10.616354 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:10.617619 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:10.618844 (+ 1225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:10.618849 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:10.618851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:10.619053 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:10.619189 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:10.619190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:10.619190 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:10.619269 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:10.619450 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:10.619453 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:10.619550 (+ 97us) client_negotiation.cc:769] Sending connection context
0926 14:56:10.619628 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 14:56:10.619731 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":75,"threads_started":1}
I20260926 14:56:10.620398 10555 heartbeater.cc:347] Connected to a master server at 127.7.37.254:46415
I20260926 14:56:10.620474 10555 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:10.620625 10555 heartbeater.cc:511] Master 127.7.37.254:46415 requested a full tablet report, sending...
I20260926 14:56:10.620879 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 10426
I20260926 14:56:10.620967 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:56:10.621116 7319 external_mini_cluster.cc:1428] Setting key 51174a066510bf0ee02349160e5b20b0
I20260926 14:56:10.621346 10364 ts_manager.cc:194] Registered new tserver with Master: b3f080c27bab478dbdf7b7d3e73c206b (127.7.37.193:38305)
I20260926 14:56:10.622144 10364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:42585
WARNING: no policy specified for kudu/127.7.37.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
I20260926 14:56:10.642227 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:46415
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:10.718185 10560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:10.718370 10560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:10.718405 10560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:10.719481 10560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:10.719523 10560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:10.719561 10560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:56:10.720934 10560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:46415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:10.721135 10560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:10.721312 10560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:10.725687 10565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:10.725694 10568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:10.725713 10566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:10.725860 10560 server_base.cc:1034] running on GCE node
I20260926 14:56:10.726047 10560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:10.726249 10560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:10.727401 10560 hybrid_clock.cc:648] HybridClock initialized: now 1790434570727359 us; error 42 us; skew 500 ppm
Sep 26 14:56:10 dist-test-slave-f9xb krb5kdc[10319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434570, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:10.729599 10560 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:56:10.729969 10560 webserver.cc:460] Webserver started at http://127.7.37.194:37869/ using document root <none> and password file <none>
I20260926 14:56:10.730137 10560 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:10.730175 10560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:10.730283 10560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:10.731186 10560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "5dc6d04a1ee840578ab3df71482d0cff"
format_stamp: "Formatted at 2026-09-26 14:56:10 on dist-test-slave-f9xb"
server_key: "d8e14afdea3ea08b625b0e96e0bcd182"
server_key_iv: "7da3772825aa25009581ffcaf438aec4"
server_key_version: "encryptionkey@0"
I20260926 14:56:10.731485 10560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "5dc6d04a1ee840578ab3df71482d0cff"
format_stamp: "Formatted at 2026-09-26 14:56:10 on dist-test-slave-f9xb"
server_key: "d8e14afdea3ea08b625b0e96e0bcd182"
server_key_iv: "7da3772825aa25009581ffcaf438aec4"
server_key_version: "encryptionkey@0"
I20260926 14:56:10.732667 10560 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 14:56:10.733276 10575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:10.733440 10560 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:56:10.733516 10560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "5dc6d04a1ee840578ab3df71482d0cff"
format_stamp: "Formatted at 2026-09-26 14:56:10 on dist-test-slave-f9xb"
server_key: "d8e14afdea3ea08b625b0e96e0bcd182"
server_key_iv: "7da3772825aa25009581ffcaf438aec4"
server_key_version: "encryptionkey@0"
I20260926 14:56:10.733575 10560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:10.763819 10560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:10.765681 10560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:10.765799 10560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:10.765976 10560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:10.766217 10560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:10.766249 10560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:10.766275 10560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:10.766296 10560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:10.771461 10560 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:36459
I20260926 14:56:10.771498 10688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:36459 every 8 connection(s)
I20260926 14:56:10.771763 10560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:56:10.775852 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 10560
I20260926 14:56:10.775929 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:56:10.776094 7319 external_mini_cluster.cc:1428] Setting key f2cb60d7c0148aa1487124bcca96fba8
Sep 26 14:56:10 dist-test-slave-f9xb krb5kdc[10319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434570, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:10.779850 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:10.772358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:57105 (local address 127.7.37.254:46415)
0926 14:56:10.772403 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:10.772405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:10.772563 (+ 158us) server_negotiation.cc:407] Connection header received
0926 14:56:10.772992 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:10.772993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:10.773001 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:10.773056 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:10.773843 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:10.774325 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:10.775221 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:10.775397 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:10.775446 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:10.777481 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:10.777482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:10.777482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:10.777489 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:10.778673 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:10.779023 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:10.779024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:10.779025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:10.779073 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:10.779276 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:10.779277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:10.779278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:10.779377 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:10.779438 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:10.779668 (+ 230us) server_negotiation.cc:299] Negotiation successful
0926 14:56:10.779765 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 14:56:10.780005 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:10.772326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:46415 (local address 127.7.37.194:57105)
0926 14:56:10.772462 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:10.772499 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:10.772917 (+ 418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:10.773101 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:10.773104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:10.773607 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:10.773773 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:10.773777 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:10.774412 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:10.774412 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:10.775156 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:10.775158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:10.775452 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:10.775453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:10.775541 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:10.775985 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:10.775993 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:10.777395 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:10.778745 (+ 1350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:10.778751 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:10.778753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:10.778961 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:10.779125 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:10.779126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:10.779126 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:10.779224 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:10.779430 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:10.779435 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:10.779559 (+ 124us) client_negotiation.cc:769] Sending connection context
0926 14:56:10.779640 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 14:56:10.779802 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
I20260926 14:56:10.780445 10689 heartbeater.cc:347] Connected to a master server at 127.7.37.254:46415
I20260926 14:56:10.780521 10689 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:10.780668 10689 heartbeater.cc:511] Master 127.7.37.254:46415 requested a full tablet report, sending...
I20260926 14:56:10.781014 10364 ts_manager.cc:194] Registered new tserver with Master: 5dc6d04a1ee840578ab3df71482d0cff (127.7.37.194:36459)
I20260926 14:56:10.781544 10364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:57105
WARNING: no policy specified for kudu/127.7.37.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
I20260926 14:56:10.797729 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:46415
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:10.871850 10694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:10.872013 10694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:10.872033 10694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:10.873121 10694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:10.873159 10694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:10.873184 10694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:56:10.874526 10694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:46415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:10.874714 10694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:10.874897 10694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 14:56:10.879312 10694 server_base.cc:1034] running on GCE node
W20260926 14:56:10.879284 10700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:10.879297 10699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:10.879298 10702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:10.879662 10694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:10.879895 10694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:10.881004 10694 hybrid_clock.cc:648] HybridClock initialized: now 1790434570880989 us; error 29 us; skew 500 ppm
Sep 26 14:56:10 dist-test-slave-f9xb krb5kdc[10319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434570, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:10.883172 10694 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:56:10.883569 10694 webserver.cc:460] Webserver started at http://127.7.37.195:35481/ using document root <none> and password file <none>
I20260926 14:56:10.883719 10694 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:10.883790 10694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:10.883883 10694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:10.884770 10694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "664f8e74e06641128194c8bf52fcf103"
format_stamp: "Formatted at 2026-09-26 14:56:10 on dist-test-slave-f9xb"
server_key: "e409b8b8ad5fff501e4c8ac7f976598e"
server_key_iv: "c487c90e3c687631e032a50c634fbaf8"
server_key_version: "encryptionkey@0"
I20260926 14:56:10.885068 10694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "664f8e74e06641128194c8bf52fcf103"
format_stamp: "Formatted at 2026-09-26 14:56:10 on dist-test-slave-f9xb"
server_key: "e409b8b8ad5fff501e4c8ac7f976598e"
server_key_iv: "c487c90e3c687631e032a50c634fbaf8"
server_key_version: "encryptionkey@0"
I20260926 14:56:10.886148 10694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:10.886739 10709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:10.886870 10694 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:10.886936 10694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "664f8e74e06641128194c8bf52fcf103"
format_stamp: "Formatted at 2026-09-26 14:56:10 on dist-test-slave-f9xb"
server_key: "e409b8b8ad5fff501e4c8ac7f976598e"
server_key_iv: "c487c90e3c687631e032a50c634fbaf8"
server_key_version: "encryptionkey@0"
I20260926 14:56:10.886996 10694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:10.897311 10694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:10.898901 10694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:10.899005 10694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:10.899175 10694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:10.899415 10694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:10.899444 10694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:10.899462 10694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:10.899484 10694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:10.904649 10694 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:32967
I20260926 14:56:10.904681 10822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:32967 every 8 connection(s)
I20260926 14:56:10.904940 10694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
Sep 26 14:56:10 dist-test-slave-f9xb krb5kdc[10319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434570, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:10.911139 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 10694
I20260926 14:56:10.911227 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:56:10.911399 7319 external_mini_cluster.cc:1428] Setting key ce2392928775d57a3466a0edd35c73a4
I20260926 14:56:10.913144 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:10.905519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:47451 (local address 127.7.37.254:46415)
0926 14:56:10.905577 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:10.905579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:10.905763 (+ 184us) server_negotiation.cc:407] Connection header received
0926 14:56:10.906178 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:10.906179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:10.906191 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:10.906239 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:10.907037 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:10.907571 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:10.908549 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:10.908796 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:10.908875 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:10.910755 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:10.910756 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:10.910757 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:10.910763 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:10.912173 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:10.912503 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:10.912503 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:10.912504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:10.912549 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:10.912703 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:10.912703 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:10.912703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:10.912795 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:10.912838 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:10.913033 (+ 195us) server_negotiation.cc:299] Negotiation successful
0926 14:56:10.913086 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 14:56:10.913280 10825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:10.905503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:46415 (local address 127.7.37.195:47451)
0926 14:56:10.905656 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:10.905684 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:10.906095 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:10.906293 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:10.906296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:10.906778 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:10.906955 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:10.906959 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:10.907640 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:10.907641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:10.908473 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:10.908476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:10.908884 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:10.908885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:10.909002 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:10.909503 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:10.909511 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:10.910676 (+ 1165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:10.912237 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:10.912243 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:10.912244 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:10.912448 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:10.912589 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:10.912589 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:10.912589 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:10.912665 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:10.912836 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:10.912838 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:10.912943 (+ 105us) client_negotiation.cc:769] Sending connection context
0926 14:56:10.913014 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 14:56:10.913113 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260926 14:56:10.913623 10823 heartbeater.cc:347] Connected to a master server at 127.7.37.254:46415
I20260926 14:56:10.913689 10823 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:10.913801 10823 heartbeater.cc:511] Master 127.7.37.254:46415 requested a full tablet report, sending...
I20260926 14:56:10.914050 10364 ts_manager.cc:194] Registered new tserver with Master: 664f8e74e06641128194c8bf52fcf103 (127.7.37.195:32967)
I20260926 14:56:10.914389 10364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:47451
I20260926 14:56:10.922834 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 14:56:10 dist-test-slave-f9xb krb5kdc[10319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434570, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 26 14:56:10 dist-test-slave-f9xb krb5kdc[10319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434570, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:10.942332 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:10.936564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36220 (local address 127.7.37.254:46415)
0926 14:56:10.936624 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:10.936626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:10.936704 (+ 78us) server_negotiation.cc:407] Connection header received
0926 14:56:10.936741 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:10.936742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:10.936749 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:10.936792 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:10.937320 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:10.937770 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:10.938453 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:10.938629 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:10.938668 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:10.940421 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:10.940422 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:10.940422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:10.940428 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:10.941567 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:10.941821 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:10.941822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:10.941822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:10.941864 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:10.941980 (+ 116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:10.941980 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:10.941981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:10.942091 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:10.942154 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:10.942224 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 14:56:10.942271 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260926 14:56:10.942550 10364 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36220
I20260926 14:56:10.942914 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 10426
I20260926 14:56:10.946660 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 10560
I20260926 14:56:10.949990 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 10694
I20260926 14:56:10.954025 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 10332
2026-09-26T14:56:10Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1713 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:56:10 dist-test-slave-f9xb krb5kdc[10835](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:56:10 dist-test-slave-f9xb krb5kdc[10835](info): set up 2 sockets
Sep 26 14:56:10 dist-test-slave-f9xb krb5kdc[10835](info): commencing operation
krb5kdc: starting...
W20260926 14:56:11.903220 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:56:11 dist-test-slave-f9xb krb5kdc[10835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:56:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:56:12.004384 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:40147
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:46829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:40147
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:12.078652 10848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:12.078824 10848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:12.078853 10848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:12.079996 10848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:12.080034 10848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:12.080051 10848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:12.080066 10848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:12.080088 10848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:56:12.081503 10848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:46829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:40147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:40147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:12.081717 10848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:12.081889 10848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:12.086095 10856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:12.086097 10854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:12.086261 10848 server_base.cc:1034] running on GCE node
W20260926 14:56:12.086419 10853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:12.086648 10848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:12.086920 10848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:12.088053 10848 hybrid_clock.cc:648] HybridClock initialized: now 1790434572088039 us; error 29 us; skew 500 ppm
Sep 26 14:56:12 dist-test-slave-f9xb krb5kdc[10835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434572, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:12.090233 10848 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:56:12.090647 10848 webserver.cc:460] Webserver started at http://127.7.37.254:39249/ using document root <none> and password file <none>
I20260926 14:56:12.090850 10848 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:12.090895 10848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:12.091017 10848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:12.091993 10848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "fa4e70ff6561454d95e84a92cd291e1a"
format_stamp: "Formatted at 2026-09-26 14:56:12 on dist-test-slave-f9xb"
server_key: "177df0cd809120dec6490a8c87a8fdf2"
server_key_iv: "77d5997df1c4545c0e253b044d1bb6a1"
server_key_version: "encryptionkey@0"
I20260926 14:56:12.092308 10848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "fa4e70ff6561454d95e84a92cd291e1a"
format_stamp: "Formatted at 2026-09-26 14:56:12 on dist-test-slave-f9xb"
server_key: "177df0cd809120dec6490a8c87a8fdf2"
server_key_iv: "77d5997df1c4545c0e253b044d1bb6a1"
server_key_version: "encryptionkey@0"
I20260926 14:56:12.093493 10848 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:12.094139 10863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:12.094290 10848 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:12.094367 10848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "fa4e70ff6561454d95e84a92cd291e1a"
format_stamp: "Formatted at 2026-09-26 14:56:12 on dist-test-slave-f9xb"
server_key: "177df0cd809120dec6490a8c87a8fdf2"
server_key_iv: "77d5997df1c4545c0e253b044d1bb6a1"
server_key_version: "encryptionkey@0"
I20260926 14:56:12.094432 10848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:12.114445 10848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:12.116241 10848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:12.116357 10848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:12.120159 10848 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:40147
I20260926 14:56:12.120193 10915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:40147 every 8 connection(s)
I20260926 14:56:12.120618 10848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:56:12.121184 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:12.123162 10916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a: Bootstrap starting.
I20260926 14:56:12.123646 10916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:12.123888 10916 log.cc:824] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:12.124483 10916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a: No bootstrap required, opened a new log
I20260926 14:56:12.125916 10916 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa4e70ff6561454d95e84a92cd291e1a" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 40147 } }
I20260926 14:56:12.126041 10916 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:12.126062 10916 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4e70ff6561454d95e84a92cd291e1a, State: Initialized, Role: FOLLOWER
I20260926 14:56:12.126159 10916 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa4e70ff6561454d95e84a92cd291e1a" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 40147 } }
I20260926 14:56:12.126230 10916 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:56:12.126260 10916 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:56:12.126292 10916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:12.127050 10916 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa4e70ff6561454d95e84a92cd291e1a" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 40147 } }
I20260926 14:56:12.127130 10916 leader_election.cc:302] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa4e70ff6561454d95e84a92cd291e1a; no voters:
I20260926 14:56:12.127241 10916 leader_election.cc:288] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:56:12.127291 10919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:12.127442 10916 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:12.127466 10919 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a [term 1 LEADER]: Becoming Leader. State: Replica: fa4e70ff6561454d95e84a92cd291e1a, State: Running, Role: LEADER
I20260926 14:56:12.127543 10919 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa4e70ff6561454d95e84a92cd291e1a" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 40147 } }
I20260926 14:56:12.127871 10921 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa4e70ff6561454d95e84a92cd291e1a. Latest consensus state: current_term: 1 leader_uuid: "fa4e70ff6561454d95e84a92cd291e1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa4e70ff6561454d95e84a92cd291e1a" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 40147 } } }
I20260926 14:56:12.127944 10921 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:12.127933 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 10848
I20260926 14:56:12.128026 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:56:12.128082 10920 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa4e70ff6561454d95e84a92cd291e1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa4e70ff6561454d95e84a92cd291e1a" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 40147 } } }
I20260926 14:56:12.128134 10920 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:12.128211 7319 external_mini_cluster.cc:1428] Setting key 3d57dae7aabb0af4ec6320a6ad82d7d8
I20260926 14:56:12.128239 10927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:56:12.128849 10927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:56:12.130165 10927 catalog_manager.cc:1347] Generated new cluster ID: d3a356801aa449a896ce02de24891d22
I20260926 14:56:12.130214 10927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 14:56:12 dist-test-slave-f9xb krb5kdc[10835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:12.137382 10937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:12.128721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56362 (local address 127.7.37.254:40147)
0926 14:56:12.128874 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:12.128883 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:12.128913 (+ 30us) server_negotiation.cc:407] Connection header received
0926 14:56:12.129026 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:12.129033 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:12.129088 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:12.129271 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:12.129878 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:12.130833 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:12.131577 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:12.132044 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:12.132152 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:12.134151 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:12.134154 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:12.134155 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:12.134165 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:12.136086 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:12.136391 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:12.136394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:12.136394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:12.136471 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:12.136636 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:12.136637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:12.136637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:12.136852 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:12.136996 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:12.137084 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 14:56:12.137205 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
I20260926 14:56:12.151572 10927 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:56:12.152086 10927 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:12.155561 10927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa4e70ff6561454d95e84a92cd291e1a: Generated new TSK 0
I20260926 14:56:12.155701 10927 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
I20260926 14:56:12.180689 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:40147
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:46829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:12.275473 10942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:12.275648 10942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:12.275676 10942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:12.276793 10942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:12.276829 10942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:12.276861 10942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:56:12.278537 10942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:46829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:40147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:12.278738 10942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:12.278926 10942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:12.283298 10948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:12.283298 10947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:12.283321 10950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:12.283501 10942 server_base.cc:1034] running on GCE node
I20260926 14:56:12.283695 10942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:12.283924 10942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:12.285069 10942 hybrid_clock.cc:648] HybridClock initialized: now 1790434572285055 us; error 28 us; skew 500 ppm
Sep 26 14:56:12 dist-test-slave-f9xb krb5kdc[10835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434572, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:12.293668 10942 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:56:12.294036 10942 webserver.cc:460] Webserver started at http://127.7.37.193:43571/ using document root <none> and password file <none>
I20260926 14:56:12.294210 10942 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:12.294255 10942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:12.294387 10942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:12.295296 10942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "f7804bedf831448196f76377d7712e1f"
format_stamp: "Formatted at 2026-09-26 14:56:12 on dist-test-slave-f9xb"
server_key: "e0bf57e431014c118e96e4a16043e760"
server_key_iv: "c06c470f3c1bce9f8d354e3baf5e1485"
server_key_version: "encryptionkey@0"
I20260926 14:56:12.295593 10942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "f7804bedf831448196f76377d7712e1f"
format_stamp: "Formatted at 2026-09-26 14:56:12 on dist-test-slave-f9xb"
server_key: "e0bf57e431014c118e96e4a16043e760"
server_key_iv: "c06c470f3c1bce9f8d354e3baf5e1485"
server_key_version: "encryptionkey@0"
I20260926 14:56:12.296851 10942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:12.297529 10957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:12.297716 10942 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:12.297791 10942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "f7804bedf831448196f76377d7712e1f"
format_stamp: "Formatted at 2026-09-26 14:56:12 on dist-test-slave-f9xb"
server_key: "e0bf57e431014c118e96e4a16043e760"
server_key_iv: "c06c470f3c1bce9f8d354e3baf5e1485"
server_key_version: "encryptionkey@0"
I20260926 14:56:12.297852 10942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:12.315104 10942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:12.317461 10942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:12.317582 10942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:12.317762 10942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:12.318054 10942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:12.318085 10942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:12.318105 10942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:12.318120 10942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:12.323226 10942 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:33303
I20260926 14:56:12.323285 11070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:33303 every 8 connection(s)
I20260926 14:56:12.323529 10942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:56:12.324852 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 10942
I20260926 14:56:12.324939 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:56:12.325095 7319 external_mini_cluster.cc:1428] Setting key ca957dce1b2b663ba4bcce8b4a69cd4a
Sep 26 14:56:12 dist-test-slave-f9xb krb5kdc[10835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434572, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.37.194@KRBTEST.COM; defaulting to no policy
I20260926 14:56:12.332683 10937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:12.324117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:39205 (local address 127.7.37.254:40147)
0926 14:56:12.324161 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:12.324164 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:12.324350 (+ 186us) server_negotiation.cc:407] Connection header received
0926 14:56:12.325255 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:12.325256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:12.325267 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:12.325315 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:12.326050 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:12.326523 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:12.327619 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:12.327970 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:12.328034 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:12.330071 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:12.330073 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:12.330073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:12.330080 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:12.331457 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:12.331820 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:12.331821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:12.331821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:12.331882 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:12.332084 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:12.332085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:12.332085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:12.332194 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:12.332248 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:12.332512 (+ 264us) server_negotiation.cc:299] Negotiation successful
0926 14:56:12.332598 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 14:56:12.332803 11073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:12.324109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:40147 (local address 127.7.37.193:39205)
0926 14:56:12.324248 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:12.324278 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:12.325149 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:12.325401 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:12.325406 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:12.325695 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:12.325956 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:12.325962 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:12.326596 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:12.326597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:12.327534 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:12.327537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:12.328042 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:12.328043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:12.328144 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:12.328598 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:12.328606 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:12.329985 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:12.331535 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:12.331540 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:12.331542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:12.331735 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:12.331940 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:12.331941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:12.331941 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:12.332031 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:12.332246 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:12.332252 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:12.332378 (+ 126us) client_negotiation.cc:769] Sending connection context
0926 14:56:12.332483 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 14:56:12.332618 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":86,"threads_started":1}
I20260926 14:56:12.333285 11071 heartbeater.cc:347] Connected to a master server at 127.7.37.254:40147
I20260926 14:56:12.333362 11071 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:12.333511 11071 heartbeater.cc:511] Master 127.7.37.254:40147 requested a full tablet report, sending...
I20260926 14:56:12.333971 10880 ts_manager.cc:194] Registered new tserver with Master: f7804bedf831448196f76377d7712e1f (127.7.37.193:33303)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.194@KRBTEST.COM" created.
I20260926 14:56:12.334621 10880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:39205
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
I20260926 14:56:12.346160 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:40147
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:46829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:12.433132 11076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:12.433301 11076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:12.433329 11076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:12.434422 11076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:12.434461 11076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:12.434495 11076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:56:12.435909 11076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:46829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:40147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:12.436108 11076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:12.436287 11076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:12.440658 11084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:12.440676 11081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:12.440730 11076 server_base.cc:1034] running on GCE node
W20260926 14:56:12.440676 11082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:12.440989 11076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:12.441200 11076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:12.442332 11076 hybrid_clock.cc:648] HybridClock initialized: now 1790434572442316 us; error 31 us; skew 500 ppm
Sep 26 14:56:12 dist-test-slave-f9xb krb5kdc[10835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434572, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:12.444526 11076 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:56:12.444859 11076 webserver.cc:460] Webserver started at http://127.7.37.194:33919/ using document root <none> and password file <none>
I20260926 14:56:12.445021 11076 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:12.445055 11076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:12.445137 11076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:12.445988 11076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "6c6a45181ac643b79f018fb5413f2153"
format_stamp: "Formatted at 2026-09-26 14:56:12 on dist-test-slave-f9xb"
server_key: "846011229138b367b4fc894fd9cbcf44"
server_key_iv: "74f2fa3c544f0d90369008eaffd18ecb"
server_key_version: "encryptionkey@0"
I20260926 14:56:12.446238 11076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "6c6a45181ac643b79f018fb5413f2153"
format_stamp: "Formatted at 2026-09-26 14:56:12 on dist-test-slave-f9xb"
server_key: "846011229138b367b4fc894fd9cbcf44"
server_key_iv: "74f2fa3c544f0d90369008eaffd18ecb"
server_key_version: "encryptionkey@0"
I20260926 14:56:12.447269 11076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 14:56:12.447842 11091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:12.447989 11076 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:56:12.448047 11076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "6c6a45181ac643b79f018fb5413f2153"
format_stamp: "Formatted at 2026-09-26 14:56:12 on dist-test-slave-f9xb"
server_key: "846011229138b367b4fc894fd9cbcf44"
server_key_iv: "74f2fa3c544f0d90369008eaffd18ecb"
server_key_version: "encryptionkey@0"
I20260926 14:56:12.448107 11076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:12.462064 11076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:12.463865 11076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:12.463958 11076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:12.464105 11076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:12.464334 11076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:12.464363 11076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:12.464386 11076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:12.464408 11076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:12.469429 11076 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:37791
I20260926 14:56:12.469511 11204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:37791 every 8 connection(s)
I20260926 14:56:12.469710 11076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
Sep 26 14:56:12 dist-test-slave-f9xb krb5kdc[10835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434572, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:12.477406 10937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:12.470272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:37977 (local address 127.7.37.254:40147)
0926 14:56:12.470327 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:12.470329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:12.470506 (+ 177us) server_negotiation.cc:407] Connection header received
0926 14:56:12.471324 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:12.471326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:12.471337 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:12.471385 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:12.471944 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:12.472441 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:12.473349 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:12.473547 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:12.473602 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:12.475328 (+ 1726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:12.475329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:12.475329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:12.475335 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:12.476375 (+ 1040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:12.476722 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:12.476723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:12.476724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:12.476780 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:12.476964 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:12.476964 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:12.476965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:12.477056 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:12.477100 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:12.477279 (+ 179us) server_negotiation.cc:299] Negotiation successful
0926 14:56:12.477344 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 14:56:12.477497 11207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:12.470248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:40147 (local address 127.7.37.194:37977)
0926 14:56:12.470383 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:12.470435 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:12.471232 (+ 797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:12.471446 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:12.471449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:12.471654 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:12.471867 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:12.471871 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:12.472514 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:12.472515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:12.473267 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:12.473269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:12.473612 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:12.473613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:12.473716 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:12.474047 (+ 331us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:12.474054 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:12.475253 (+ 1199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:12.476432 (+ 1179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:12.476454 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:12.476456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:12.476642 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:12.476835 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:12.476835 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:12.476836 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:12.476911 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:12.477096 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:12.477098 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:12.477192 (+ 94us) client_negotiation.cc:769] Sending connection context
0926 14:56:12.477260 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 14:56:12.477356 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":70,"threads_started":1}
I20260926 14:56:12.477852 11205 heartbeater.cc:347] Connected to a master server at 127.7.37.254:40147
I20260926 14:56:12.477936 11205 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:12.478070 11205 heartbeater.cc:511] Master 127.7.37.254:40147 requested a full tablet report, sending...
I20260926 14:56:12.478351 10880 ts_manager.cc:194] Registered new tserver with Master: 6c6a45181ac643b79f018fb5413f2153 (127.7.37.194:37791)
I20260926 14:56:12.478694 10880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:37977
I20260926 14:56:12.479683 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 11076
I20260926 14:56:12.479802 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:56:12.479961 7319 external_mini_cluster.cc:1428] Setting key ae4a3b08bb12994d9ed6a365f3e1e56e
WARNING: no policy specified for kudu/127.7.37.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
I20260926 14:56:12.500608 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:40147
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:46829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:12.574641 11210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:12.574800 11210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:12.574819 11210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:12.575913 11210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:12.575948 11210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:12.575975 11210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:56:12.577349 11210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:46829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:40147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:12.577546 11210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:12.577719 11210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:12.581967 11216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:12.582056 11215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:12.581967 11218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:12.582129 11210 server_base.cc:1034] running on GCE node
I20260926 14:56:12.582315 11210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:12.582517 11210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:12.583658 11210 hybrid_clock.cc:648] HybridClock initialized: now 1790434572583643 us; error 27 us; skew 500 ppm
Sep 26 14:56:12 dist-test-slave-f9xb krb5kdc[10835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434572, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:12.585739 11210 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:56:12.586097 11210 webserver.cc:460] Webserver started at http://127.7.37.195:36521/ using document root <none> and password file <none>
I20260926 14:56:12.586243 11210 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:12.586277 11210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:12.586355 11210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:12.587177 11210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "063b569a073d492aa33426c72f1bf15e"
format_stamp: "Formatted at 2026-09-26 14:56:12 on dist-test-slave-f9xb"
server_key: "a704111ab307cb022c095f4c58d2f24b"
server_key_iv: "62983b4f18061f92f12159424abaa3e1"
server_key_version: "encryptionkey@0"
I20260926 14:56:12.587432 11210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "063b569a073d492aa33426c72f1bf15e"
format_stamp: "Formatted at 2026-09-26 14:56:12 on dist-test-slave-f9xb"
server_key: "a704111ab307cb022c095f4c58d2f24b"
server_key_iv: "62983b4f18061f92f12159424abaa3e1"
server_key_version: "encryptionkey@0"
I20260926 14:56:12.588518 11210 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:12.589097 11225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:12.589237 11210 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:12.589306 11210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "063b569a073d492aa33426c72f1bf15e"
format_stamp: "Formatted at 2026-09-26 14:56:12 on dist-test-slave-f9xb"
server_key: "a704111ab307cb022c095f4c58d2f24b"
server_key_iv: "62983b4f18061f92f12159424abaa3e1"
server_key_version: "encryptionkey@0"
I20260926 14:56:12.589354 11210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:12.596338 11210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:12.598160 11210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:12.598251 11210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:12.598402 11210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:12.598608 11210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:12.598634 11210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:12.598652 11210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:12.598668 11210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:12.603590 11210 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:40023
I20260926 14:56:12.603644 11338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:40023 every 8 connection(s)
I20260926 14:56:12.603892 11210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
Sep 26 14:56:12 dist-test-slave-f9xb krb5kdc[10835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434572, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:12.611274 10937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:12.604425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:58253 (local address 127.7.37.254:40147)
0926 14:56:12.604476 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:12.604478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:12.604644 (+ 166us) server_negotiation.cc:407] Connection header received
0926 14:56:12.605406 (+ 762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:12.605407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:12.605418 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:12.605464 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:12.605965 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:12.606442 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:12.607325 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:12.607527 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:12.607579 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:12.609311 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:12.609312 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:12.609312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:12.609318 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:12.610294 (+ 976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:12.610622 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:12.610623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:12.610623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:12.610663 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:12.610839 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:12.610839 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:12.610839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:12.610931 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:12.610972 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:12.611166 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 14:56:12.611216 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 14:56:12.611400 11341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:12.604403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:40147 (local address 127.7.37.195:58253)
0926 14:56:12.604551 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:12.604579 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:12.605321 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:12.605522 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:12.605526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:12.605735 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:12.605891 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:12.605894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:12.606512 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:12.606513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:12.607253 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:12.607255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:12.607588 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:12.607589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:12.607690 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:12.608057 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:12.608064 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:12.609238 (+ 1174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:12.610356 (+ 1118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:12.610361 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:12.610362 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:12.610570 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:12.610714 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:12.610715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:12.610715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:12.610797 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:12.610971 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:12.610973 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:12.611065 (+ 92us) client_negotiation.cc:769] Sending connection context
0926 14:56:12.611147 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 14:56:12.611248 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260926 14:56:12.611835 11339 heartbeater.cc:347] Connected to a master server at 127.7.37.254:40147
I20260926 14:56:12.611904 11339 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:12.612023 11339 heartbeater.cc:511] Master 127.7.37.254:40147 requested a full tablet report, sending...
I20260926 14:56:12.612306 10880 ts_manager.cc:194] Registered new tserver with Master: 063b569a073d492aa33426c72f1bf15e (127.7.37.195:40023)
I20260926 14:56:12.612656 10880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:58253
I20260926 14:56:12.613997 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 11210
I20260926 14:56:12.614094 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:56:12.614221 7319 external_mini_cluster.cc:1428] Setting key 8d2e3b30992de1280623756672f8d861
I20260926 14:56:12.614892 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 14:56:12 dist-test-slave-f9xb krb5kdc[10835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 14:56:12 dist-test-slave-f9xb krb5kdc[10835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.193@KRBTEST.COM
I20260926 14:56:12.635187 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:12.628398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52112 (local address 127.7.37.193:33303)
0926 14:56:12.628592 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:12.628597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:12.628606 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:56:12.628627 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:12.628629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:12.628642 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:12.628714 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:12.629345 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:12.630028 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:12.630759 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:12.631061 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:12.631137 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:12.633133 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:12.633139 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:12.633139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:12.633145 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:12.634337 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:12.634639 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:12.634641 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:12.634641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:12.634684 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:12.634809 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:12.634809 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:12.634810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:12.634911 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:12.635001 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:12.635063 (+ 62us) server_negotiation.cc:299] Negotiation successful
0926 14:56:12.635119 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":63,"threads_started":1}
Sep 26 14:56:12 dist-test-slave-f9xb krb5kdc[10835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:12.641894 10937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:12.636216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56374 (local address 127.7.37.254:40147)
0926 14:56:12.636264 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:12.636266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:12.636320 (+ 54us) server_negotiation.cc:407] Connection header received
0926 14:56:12.636382 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:12.636383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:12.636389 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:12.636434 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:12.636825 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:12.637252 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:12.637970 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:12.638143 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:12.638188 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:12.640103 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:12.640103 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:12.640104 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:12.640109 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:12.641094 (+ 985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:12.641369 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:12.641370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:12.641370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:12.641408 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:12.641545 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:12.641545 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:12.641545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:12.641648 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:12.641708 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:12.641792 (+ 84us) server_negotiation.cc:299] Negotiation successful
0926 14:56:12.641840 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260926 14:56:12.642102 10880 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56374
I20260926 14:56:12.642400 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 10942
I20260926 14:56:12.646512 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 11076
I20260926 14:56:12.649820 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 11210
I20260926 14:56:12.653050 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 10848
2026-09-26T14:56:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1699 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:56:12 dist-test-slave-f9xb krb5kdc[11351](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:56:12 dist-test-slave-f9xb krb5kdc[11351](info): set up 2 sockets
Sep 26 14:56:12 dist-test-slave-f9xb krb5kdc[11351](info): commencing operation
krb5kdc: starting...
W20260926 14:56:13.595219 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.004s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:56:13 dist-test-slave-f9xb krb5kdc[11351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:56:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:56:13.698000 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:42929
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:40879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:42929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:13.772027 11364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:13.772204 11364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:13.772233 11364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:13.773329 11364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:13.773367 11364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:13.773386 11364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:13.773402 11364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:13.773417 11364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:56:13.774833 11364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:40879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:42929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:42929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:13.775044 11364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:13.775230 11364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:13.779467 11369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:13.779498 11372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:13.779589 11364 server_base.cc:1034] running on GCE node
W20260926 14:56:13.779511 11370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:13.779873 11364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:13.780138 11364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:13.781265 11364 hybrid_clock.cc:648] HybridClock initialized: now 1790434573781253 us; error 27 us; skew 500 ppm
Sep 26 14:56:13 dist-test-slave-f9xb krb5kdc[11351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434573, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:13.783389 11364 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:56:13.783668 11364 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:13.783713 11364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:13.783845 11364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:13.784726 11364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "279ca7ba271b4177a95f20f743686394"
format_stamp: "Formatted at 2026-09-26 14:56:13 on dist-test-slave-f9xb"
server_key: "a33f64228bfe4ef6f51ff38f7cd4ae2b"
server_key_iv: "10f11da9c8c3c696b2ad3ef1d6f0d7fc"
server_key_version: "encryptionkey@0"
I20260926 14:56:13.784986 11364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "279ca7ba271b4177a95f20f743686394"
format_stamp: "Formatted at 2026-09-26 14:56:13 on dist-test-slave-f9xb"
server_key: "a33f64228bfe4ef6f51ff38f7cd4ae2b"
server_key_iv: "10f11da9c8c3c696b2ad3ef1d6f0d7fc"
server_key_version: "encryptionkey@0"
I20260926 14:56:13.786063 11364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 14:56:13.786593 11378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:13.786725 11364 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:13.786801 11364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "279ca7ba271b4177a95f20f743686394"
format_stamp: "Formatted at 2026-09-26 14:56:13 on dist-test-slave-f9xb"
server_key: "a33f64228bfe4ef6f51ff38f7cd4ae2b"
server_key_iv: "10f11da9c8c3c696b2ad3ef1d6f0d7fc"
server_key_version: "encryptionkey@0"
I20260926 14:56:13.786850 11364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:13.804205 11364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:13.805994 11364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:13.806087 11364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:13.809376 11364 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:42929
I20260926 14:56:13.809407 11430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:42929 every 8 connection(s)
I20260926 14:56:13.809618 11364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:56:13.810384 11431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:13.811381 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 11364
I20260926 14:56:13.811493 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:56:13.811659 7319 external_mini_cluster.cc:1428] Setting key 89154e08a1d464dcdf35d9a556fe8401
I20260926 14:56:13.812654 11431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394: Bootstrap starting.
I20260926 14:56:13.813164 11431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:13.813405 11431 log.cc:824] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:13.813982 11431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394: No bootstrap required, opened a new log
I20260926 14:56:13.815512 11431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279ca7ba271b4177a95f20f743686394" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42929 } }
I20260926 14:56:13.815627 11431 raft_consensus.cc:374] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:13.815657 11431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 279ca7ba271b4177a95f20f743686394, State: Initialized, Role: FOLLOWER
I20260926 14:56:13.815790 11431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279ca7ba271b4177a95f20f743686394" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42929 } }
I20260926 14:56:13.815856 11431 raft_consensus.cc:388] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:56:13.815884 11431 raft_consensus.cc:482] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:56:13.815936 11431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:13.816376 11431 raft_consensus.cc:504] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279ca7ba271b4177a95f20f743686394" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42929 } }
I20260926 14:56:13.816461 11431 leader_election.cc:302] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 279ca7ba271b4177a95f20f743686394; no voters:
I20260926 14:56:13.816593 11431 leader_election.cc:288] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:56:13.816617 11436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:13.816710 11436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394 [term 1 LEADER]: Becoming Leader. State: Replica: 279ca7ba271b4177a95f20f743686394, State: Running, Role: LEADER
I20260926 14:56:13.816789 11436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279ca7ba271b4177a95f20f743686394" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42929 } }
I20260926 14:56:13.816812 11431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:13.817199 11437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "279ca7ba271b4177a95f20f743686394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279ca7ba271b4177a95f20f743686394" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42929 } } }
I20260926 14:56:13.817209 11438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 279ca7ba271b4177a95f20f743686394. Latest consensus state: current_term: 1 leader_uuid: "279ca7ba271b4177a95f20f743686394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279ca7ba271b4177a95f20f743686394" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42929 } } }
I20260926 14:56:13.817289 11437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:13.817296 11438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394 [sys.catalog]: This master's current role is: LEADER
Sep 26 14:56:13 dist-test-slave-f9xb krb5kdc[11351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
W20260926 14:56:13.818552 11452 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 14:56:13.818598 11452 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 14:56:13.818663 11441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:56:13.818810 11441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:56:13.820223 11441 catalog_manager.cc:1347] Generated new cluster ID: 84d514a9a60348468084c7e809e7bf6c
I20260926 14:56:13.820268 11441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:56:13.821368 11434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:13.812148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49754 (local address 127.7.37.254:42929)
0926 14:56:13.812309 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:13.812316 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:13.812337 (+ 21us) server_negotiation.cc:407] Connection header received
0926 14:56:13.812687 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:13.812693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:13.812748 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:13.812875 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:13.813478 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:13.814457 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:13.815161 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:13.815536 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:13.815622 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:13.817824 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:13.817828 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:13.817829 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:13.817840 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:13.819540 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:13.819905 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:13.819907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:13.819908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:13.820005 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:13.820238 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:13.820239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:13.820239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:13.820526 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:13.820640 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:13.821051 (+ 411us) server_negotiation.cc:299] Negotiation successful
0926 14:56:13.821171 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260926 14:56:13.833271 11441 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:56:13.833714 11441 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:13.837602 11441 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394: Generated new TSK 0
I20260926 14:56:13.837726 11441 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
I20260926 14:56:13.858428 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42929
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:40879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:13.932202 11457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:13.932377 11457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:13.932406 11457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:13.933504 11457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:13.933542 11457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:13.933576 11457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:56:13.934973 11457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:40879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:13.935176 11457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:13.935359 11457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:13.939565 11463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:13.939589 11465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:13.939632 11457 server_base.cc:1034] running on GCE node
W20260926 14:56:13.939550 11462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:13.940037 11457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:13.940249 11457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:13.941385 11457 hybrid_clock.cc:648] HybridClock initialized: now 1790434573941374 us; error 23 us; skew 500 ppm
Sep 26 14:56:13 dist-test-slave-f9xb krb5kdc[11351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434573, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:13.943728 11457 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:56:13.944048 11457 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:13.944100 11457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:13.944219 11457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:13.945138 11457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "ab64a1a3646a40e0baec01867840004c"
format_stamp: "Formatted at 2026-09-26 14:56:13 on dist-test-slave-f9xb"
server_key: "8916105f01715bbc2f0156afc4c2686b"
server_key_iv: "8bd5c292959de36ca45b180b4ca09c5c"
server_key_version: "encryptionkey@0"
I20260926 14:56:13.945479 11457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "ab64a1a3646a40e0baec01867840004c"
format_stamp: "Formatted at 2026-09-26 14:56:13 on dist-test-slave-f9xb"
server_key: "8916105f01715bbc2f0156afc4c2686b"
server_key_iv: "8bd5c292959de36ca45b180b4ca09c5c"
server_key_version: "encryptionkey@0"
I20260926 14:56:13.946719 11457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260926 14:56:13.947394 11471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:13.947542 11457 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:13.947615 11457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "ab64a1a3646a40e0baec01867840004c"
format_stamp: "Formatted at 2026-09-26 14:56:13 on dist-test-slave-f9xb"
server_key: "8916105f01715bbc2f0156afc4c2686b"
server_key_iv: "8bd5c292959de36ca45b180b4ca09c5c"
server_key_version: "encryptionkey@0"
I20260926 14:56:13.947670 11457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:13.980742 11457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:13.982648 11457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:13.982767 11457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:13.982945 11457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:13.983214 11457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:13.983249 11457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:13.983275 11457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:13.983296 11457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:13.988387 11457 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:46169
I20260926 14:56:13.988431 11584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:46169 every 8 connection(s)
I20260926 14:56:13.988642 11457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:56:13.992285 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 11457
I20260926 14:56:13.992378 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:56:13.992574 7319 external_mini_cluster.cc:1428] Setting key a33c3a752b5b7196052b7c85eee84241
Sep 26 14:56:13 dist-test-slave-f9xb krb5kdc[11351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434573, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:13.997108 11434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:13.989212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:52383 (local address 127.7.37.254:42929)
0926 14:56:13.989256 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:13.989258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:13.989432 (+ 174us) server_negotiation.cc:407] Connection header received
0926 14:56:13.989872 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:13.989873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:13.989883 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:13.989937 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:13.990817 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:13.991309 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:13.992318 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:13.992609 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:13.992679 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:13.994541 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:13.994542 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:13.994543 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:13.994549 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:13.995890 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:13.996239 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:13.996240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:13.996240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:13.996300 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:13.996520 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:13.996521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:13.996521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:13.996642 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:13.996709 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:13.996937 (+ 228us) server_negotiation.cc:299] Negotiation successful
0926 14:56:13.997015 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 14:56:13.997145 11587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:13.989210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42929 (local address 127.7.37.193:52383)
0926 14:56:13.989335 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:13.989366 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:13.989767 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:13.989996 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:13.989999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:13.990266 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:13.990737 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:13.990741 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:13.991383 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:13.991384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:13.992242 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:13.992244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:13.992684 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:13.992684 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:13.992812 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:13.993233 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:13.993243 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:13.994462 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:13.995956 (+ 1494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:13.995962 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:13.995963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:13.996163 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:13.996365 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:13.996366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:13.996366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:13.996467 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:13.996705 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:13.996709 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:13.996812 (+ 103us) client_negotiation.cc:769] Sending connection context
0926 14:56:13.996895 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 14:56:13.997004 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":68,"threads_started":1}
I20260926 14:56:13.997605 11585 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42929
I20260926 14:56:13.997679 11585 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:13.997797 11585 heartbeater.cc:511] Master 127.7.37.254:42929 requested a full tablet report, sending...
I20260926 14:56:13.998299 11395 ts_manager.cc:194] Registered new tserver with Master: ab64a1a3646a40e0baec01867840004c (127.7.37.193:46169)
I20260926 14:56:13.999148 11395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:52383
WARNING: no policy specified for kudu/127.7.37.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
I20260926 14:56:14.015568 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42929
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:40879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:14.091362 11590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:14.091529 11590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:14.091558 11590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:14.092669 11590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:14.092705 11590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:14.092737 11590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:56:14.094115 11590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:40879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:14.094314 11590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:14.094501 11590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:14.098869 11598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:14.099016 11590 server_base.cc:1034] running on GCE node
W20260926 14:56:14.098901 11595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:14.098879 11596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:14.099299 11590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:14.099491 11590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:14.100625 11590 hybrid_clock.cc:648] HybridClock initialized: now 1790434574100612 us; error 27 us; skew 500 ppm
Sep 26 14:56:14 dist-test-slave-f9xb krb5kdc[11351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434574, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:14.102880 11590 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:56:14.103137 11590 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:14.103185 11590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:14.103297 11590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:14.104226 11590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "4a6c5100deae434bbea5b2a0e6b6c00b"
format_stamp: "Formatted at 2026-09-26 14:56:14 on dist-test-slave-f9xb"
server_key: "eec8b0675d9297dce98248e5f1ae58c0"
server_key_iv: "fe4fe0675ce246b6eb8bf24fe42eaf74"
server_key_version: "encryptionkey@0"
I20260926 14:56:14.104518 11590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "4a6c5100deae434bbea5b2a0e6b6c00b"
format_stamp: "Formatted at 2026-09-26 14:56:14 on dist-test-slave-f9xb"
server_key: "eec8b0675d9297dce98248e5f1ae58c0"
server_key_iv: "fe4fe0675ce246b6eb8bf24fe42eaf74"
server_key_version: "encryptionkey@0"
I20260926 14:56:14.105598 11590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:14.106211 11604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:14.106375 11590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:14.106451 11590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "4a6c5100deae434bbea5b2a0e6b6c00b"
format_stamp: "Formatted at 2026-09-26 14:56:14 on dist-test-slave-f9xb"
server_key: "eec8b0675d9297dce98248e5f1ae58c0"
server_key_iv: "fe4fe0675ce246b6eb8bf24fe42eaf74"
server_key_version: "encryptionkey@0"
I20260926 14:56:14.106505 11590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:14.117830 11590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:14.119793 11590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:14.119918 11590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:14.120101 11590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:14.120360 11590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:14.120391 11590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:14.120416 11590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:14.120438 11590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:14.125432 11590 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:36345
I20260926 14:56:14.125488 11717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:36345 every 8 connection(s)
I20260926 14:56:14.125684 11590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:56:14.129339 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 11590
I20260926 14:56:14.129443 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:56:14.129596 7319 external_mini_cluster.cc:1428] Setting key c4e29a4d77b8bdf6c3a862cfdb8472ea
Sep 26 14:56:14 dist-test-slave-f9xb krb5kdc[11351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434574, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:14.133440 11434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.126225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:52379 (local address 127.7.37.254:42929)
0926 14:56:14.126280 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:14.126282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:14.126449 (+ 167us) server_negotiation.cc:407] Connection header received
0926 14:56:14.126864 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:14.126865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:14.126873 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:14.126921 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:14.127685 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.128177 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.129049 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.129220 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.129272 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:14.131141 (+ 1869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:14.131143 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:14.131143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:14.131150 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:14.132313 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:14.132644 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:14.132645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:14.132645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:14.132694 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:14.132889 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:14.132890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:14.132890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:14.132989 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:14.133041 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:14.133272 (+ 231us) server_negotiation.cc:299] Negotiation successful
0926 14:56:14.133353 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 14:56:14.133612 11720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.126202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42929 (local address 127.7.37.194:52379)
0926 14:56:14.126340 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:14.126385 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:14.126795 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:14.126976 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:14.126979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:14.127190 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:14.127624 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:14.127628 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.128236 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.128237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:14.128992 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:14.128994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.129296 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.129298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:14.129407 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:14.129856 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:14.129864 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:14.131047 (+ 1183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:14.132381 (+ 1334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:14.132386 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:14.132388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:14.132582 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:14.132744 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:14.132745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:14.132745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:14.132838 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:14.133040 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:14.133046 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:14.133165 (+ 119us) client_negotiation.cc:769] Sending connection context
0926 14:56:14.133245 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 14:56:14.133378 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
I20260926 14:56:14.134171 11718 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42929
I20260926 14:56:14.134244 11718 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:14.134369 11718 heartbeater.cc:511] Master 127.7.37.254:42929 requested a full tablet report, sending...
I20260926 14:56:14.134690 11395 ts_manager.cc:194] Registered new tserver with Master: 4a6c5100deae434bbea5b2a0e6b6c00b (127.7.37.194:36345)
I20260926 14:56:14.135170 11395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:52379
WARNING: no policy specified for kudu/127.7.37.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
I20260926 14:56:14.150503 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42929
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:40879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:14.225013 11723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:14.225180 11723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:14.225200 11723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:14.226274 11723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:14.226307 11723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:14.226331 11723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:56:14.227687 11723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:40879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:14.227856 11723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:14.228019 11723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:14.232278 11731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:14.232291 11729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:14.232355 11728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:14.232582 11723 server_base.cc:1034] running on GCE node
I20260926 14:56:14.232784 11723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:14.232988 11723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:14.234122 11723 hybrid_clock.cc:648] HybridClock initialized: now 1790434574234110 us; error 24 us; skew 500 ppm
Sep 26 14:56:14 dist-test-slave-f9xb krb5kdc[11351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434574, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:14.236263 11723 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:56:14.236550 11723 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:14.236598 11723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:14.236714 11723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:14.237605 11723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "dd55cb52c56c47bab3d2c439d32580a5"
format_stamp: "Formatted at 2026-09-26 14:56:14 on dist-test-slave-f9xb"
server_key: "9467699752104f03fc4171331656617a"
server_key_iv: "b93ed8d64cb75fea78ccf678b24503e2"
server_key_version: "encryptionkey@0"
I20260926 14:56:14.237900 11723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "dd55cb52c56c47bab3d2c439d32580a5"
format_stamp: "Formatted at 2026-09-26 14:56:14 on dist-test-slave-f9xb"
server_key: "9467699752104f03fc4171331656617a"
server_key_iv: "b93ed8d64cb75fea78ccf678b24503e2"
server_key_version: "encryptionkey@0"
I20260926 14:56:14.238933 11723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:14.239523 11737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:14.239676 11723 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:14.239812 11723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "dd55cb52c56c47bab3d2c439d32580a5"
format_stamp: "Formatted at 2026-09-26 14:56:14 on dist-test-slave-f9xb"
server_key: "9467699752104f03fc4171331656617a"
server_key_iv: "b93ed8d64cb75fea78ccf678b24503e2"
server_key_version: "encryptionkey@0"
I20260926 14:56:14.239876 11723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:14.251729 11723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:14.253280 11723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:14.253389 11723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:14.253561 11723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:14.253827 11723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:14.253865 11723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:14.253890 11723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:14.253913 11723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:14.258849 11723 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:38683
I20260926 14:56:14.258901 11850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:38683 every 8 connection(s)
I20260926 14:56:14.259096 11723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
Sep 26 14:56:14 dist-test-slave-f9xb krb5kdc[11351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434574, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:14.263998 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 11723
I20260926 14:56:14.264088 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:56:14.264245 7319 external_mini_cluster.cc:1428] Setting key be4d43bd783a6529d66b5b193c7c4b50
I20260926 14:56:14.266462 11434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.259609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:33203 (local address 127.7.37.254:42929)
0926 14:56:14.259651 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:14.259653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:14.259843 (+ 190us) server_negotiation.cc:407] Connection header received
0926 14:56:14.260217 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:14.260218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:14.260227 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:14.260272 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:14.261017 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.261495 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.262358 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.262560 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.262605 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:14.264281 (+ 1676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:14.264283 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:14.264283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:14.264290 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:14.265541 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:14.265846 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:14.265846 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:14.265847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:14.265887 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:14.266059 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:14.266059 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:14.266060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:14.266142 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:14.266183 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:14.266355 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 14:56:14.266403 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 14:56:14.266616 11853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.259606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42929 (local address 127.7.37.195:33203)
0926 14:56:14.259732 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:14.259784 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:14.260148 (+ 364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:14.260316 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:14.260319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:14.260538 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:14.260955 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:14.260959 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.261555 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.261556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:14.262294 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:14.262296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.262613 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.262614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:14.262699 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:14.263025 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:14.263032 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:14.264202 (+ 1170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:14.265602 (+ 1400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:14.265607 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:14.265609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:14.265793 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:14.265928 (+ 135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:14.265929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:14.265929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:14.266019 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:14.266181 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:14.266183 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:14.266275 (+ 92us) client_negotiation.cc:769] Sending connection context
0926 14:56:14.266342 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 14:56:14.266439 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":72,"threads_started":1}
I20260926 14:56:14.266999 11851 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42929
I20260926 14:56:14.267066 11851 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:14.267185 11851 heartbeater.cc:511] Master 127.7.37.254:42929 requested a full tablet report, sending...
I20260926 14:56:14.267472 11395 ts_manager.cc:194] Registered new tserver with Master: dd55cb52c56c47bab3d2c439d32580a5 (127.7.37.195:38683)
I20260926 14:56:14.267863 11395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:33203
I20260926 14:56:14.275594 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 14:56:14.281566 11434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.276758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49770 (local address 127.7.37.254:42929)
0926 14:56:14.276812 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:14.276813 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:14.276894 (+ 81us) server_negotiation.cc:407] Connection header received
0926 14:56:14.276929 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:14.276930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:14.276937 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:14.276987 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:14.277466 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.277887 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.278556 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.278714 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.278752 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:14.279706 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:14.279707 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:14.279707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:14.279712 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:14.280826 (+ 1114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:14.281075 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:14.281075 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:14.281075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:14.281114 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:14.281215 (+ 101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:14.281216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:14.281216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:14.281320 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:14.281390 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:14.281462 (+ 72us) server_negotiation.cc:299] Negotiation successful
0926 14:56:14.281513 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 14:56:14.282670 11395 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:56:14.284188 11395 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:56:14.289824 11860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.285844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:46169 (local address 127.0.0.1:38482)
0926 14:56:14.286231 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:14.286267 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:14.286327 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:14.286529 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:14.286532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:14.286534 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:14.286658 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:14.286659 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.287584 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.287585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:14.288716 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:14.288719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.289559 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.289559 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:14.289631 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:14.289644 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 14:56:14.289710 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 14:56:14.289757 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":52,"thread_start_us":48,"threads_started":1}
I20260926 14:56:14.289845 11862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.286071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38482 (local address 127.7.37.193:46169)
0926 14:56:14.286321 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:14.286325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:14.286334 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:56:14.286406 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:14.286408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:14.286423 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:14.286515 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:14.286767 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.287502 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.288796 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.289475 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.289537 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:14.289586 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:14.289731 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 14:56:14.289770 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":70,"threads_started":1}
I20260926 14:56:14.290158 11864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.285862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:36345 (local address 127.0.0.1:53802)
0926 14:56:14.286250 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:14.286267 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:14.286327 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:14.286606 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:14.286607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:14.286608 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:14.286671 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:14.286673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.287922 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.287923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:14.289118 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:14.289120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.289949 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.289950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:14.290011 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:14.290014 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:14.290061 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 14:56:14.290100 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":50,"thread_start_us":70,"threads_started":1}
I20260926 14:56:14.290197 11863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.286074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53802 (local address 127.7.37.194:36345)
0926 14:56:14.286242 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:14.286247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:14.286400 (+ 153us) server_negotiation.cc:407] Connection header received
0926 14:56:14.286441 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:14.286443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:14.286466 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:14.286530 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:14.286747 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.287829 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.289206 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.289808 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.289865 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:14.289918 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:14.290090 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 14:56:14.290127 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":77,"threads_started":1}
I20260926 14:56:14.290323 11861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.285999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56340 (local address 127.7.37.195:38683)
0926 14:56:14.286157 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:14.286161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:14.286327 (+ 166us) server_negotiation.cc:407] Connection header received
0926 14:56:14.286439 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:14.286442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:14.286456 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:14.286523 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:14.286749 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.287821 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.289205 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.289916 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.289975 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:14.290024 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:14.290222 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 14:56:14.290258 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260926 14:56:14.290508 11859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.286140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:38683 (local address 127.0.0.1:56340)
0926 14:56:14.286264 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:14.286272 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:14.286353 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:14.286591 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:14.286592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:14.286595 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:14.286671 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:14.286674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.287924 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.287924 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:14.289119 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:14.289121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.290055 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.290056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:14.290133 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:14.290137 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:56:14.290383 (+ 246us) client_negotiation.cc:240] Negotiation successful
0926 14:56:14.290428 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":62,"thread_start_us":75,"threads_started":1}
I20260926 14:56:14.290726 11519 tablet_service.cc:1467] Processing CreateTablet for tablet b0745260aec04822b3be9ee86f30a507 (DEFAULT_TABLE table=test-table [id=af07600220384381a0ba2b8e6a572860]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:14.290902 11652 tablet_service.cc:1467] Processing CreateTablet for tablet b0745260aec04822b3be9ee86f30a507 (DEFAULT_TABLE table=test-table [id=af07600220384381a0ba2b8e6a572860]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:14.291035 11519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0745260aec04822b3be9ee86f30a507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:14.291091 11785 tablet_service.cc:1467] Processing CreateTablet for tablet b0745260aec04822b3be9ee86f30a507 (DEFAULT_TABLE table=test-table [id=af07600220384381a0ba2b8e6a572860]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:14.291224 11652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0745260aec04822b3be9ee86f30a507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:14.291344 11785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0745260aec04822b3be9ee86f30a507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:14.293448 11865 tablet_bootstrap.cc:492] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b: Bootstrap starting.
I20260926 14:56:14.294050 11867 tablet_bootstrap.cc:492] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5: Bootstrap starting.
I20260926 14:56:14.294382 11865 tablet_bootstrap.cc:654] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:14.294602 11867 tablet_bootstrap.cc:654] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:14.294938 11865 log.cc:824] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:14.295120 11867 log.cc:824] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:14.295396 11866 tablet_bootstrap.cc:492] T b0745260aec04822b3be9ee86f30a507 P ab64a1a3646a40e0baec01867840004c: Bootstrap starting.
I20260926 14:56:14.295643 11865 tablet_bootstrap.cc:492] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b: No bootstrap required, opened a new log
I20260926 14:56:14.295707 11865 ts_tablet_manager.cc:1397] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:56:14.295799 11867 tablet_bootstrap.cc:492] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5: No bootstrap required, opened a new log
I20260926 14:56:14.295861 11867 ts_tablet_manager.cc:1397] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:56:14.295979 11866 tablet_bootstrap.cc:654] T b0745260aec04822b3be9ee86f30a507 P ab64a1a3646a40e0baec01867840004c: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:14.296492 11866 log.cc:824] T b0745260aec04822b3be9ee86f30a507 P ab64a1a3646a40e0baec01867840004c: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:14.297215 11866 tablet_bootstrap.cc:492] T b0745260aec04822b3be9ee86f30a507 P ab64a1a3646a40e0baec01867840004c: No bootstrap required, opened a new log
I20260926 14:56:14.297276 11866 ts_tablet_manager.cc:1397] T b0745260aec04822b3be9ee86f30a507 P ab64a1a3646a40e0baec01867840004c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:56:14.297399 11867 raft_consensus.cc:348] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab64a1a3646a40e0baec01867840004c" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 46169 } } peers { permanent_uuid: "dd55cb52c56c47bab3d2c439d32580a5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38683 } } peers { permanent_uuid: "4a6c5100deae434bbea5b2a0e6b6c00b" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 36345 } }
I20260926 14:56:14.297523 11867 raft_consensus.cc:374] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:14.297555 11867 raft_consensus.cc:729] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd55cb52c56c47bab3d2c439d32580a5, State: Initialized, Role: FOLLOWER
I20260926 14:56:14.297516 11865 raft_consensus.cc:348] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab64a1a3646a40e0baec01867840004c" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 46169 } } peers { permanent_uuid: "dd55cb52c56c47bab3d2c439d32580a5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38683 } } peers { permanent_uuid: "4a6c5100deae434bbea5b2a0e6b6c00b" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 36345 } }
I20260926 14:56:14.297667 11865 raft_consensus.cc:374] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:14.297704 11865 raft_consensus.cc:729] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a6c5100deae434bbea5b2a0e6b6c00b, State: Initialized, Role: FOLLOWER
I20260926 14:56:14.297672 11867 consensus_queue.cc:260] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab64a1a3646a40e0baec01867840004c" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 46169 } } peers { permanent_uuid: "dd55cb52c56c47bab3d2c439d32580a5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38683 } } peers { permanent_uuid: "4a6c5100deae434bbea5b2a0e6b6c00b" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 36345 } }
I20260926 14:56:14.297825 11865 consensus_queue.cc:260] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab64a1a3646a40e0baec01867840004c" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 46169 } } peers { permanent_uuid: "dd55cb52c56c47bab3d2c439d32580a5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38683 } } peers { permanent_uuid: "4a6c5100deae434bbea5b2a0e6b6c00b" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 36345 } }
I20260926 14:56:14.297896 11851 heartbeater.cc:503] Master 127.7.37.254:42929 was elected leader, sending a full tablet report...
I20260926 14:56:14.298075 11718 heartbeater.cc:503] Master 127.7.37.254:42929 was elected leader, sending a full tablet report...
I20260926 14:56:14.298076 11865 ts_tablet_manager.cc:1428] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 14:56:14.298128 11867 ts_tablet_manager.cc:1428] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:56:14.299165 11866 raft_consensus.cc:348] T b0745260aec04822b3be9ee86f30a507 P ab64a1a3646a40e0baec01867840004c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab64a1a3646a40e0baec01867840004c" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 46169 } } peers { permanent_uuid: "dd55cb52c56c47bab3d2c439d32580a5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38683 } } peers { permanent_uuid: "4a6c5100deae434bbea5b2a0e6b6c00b" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 36345 } }
I20260926 14:56:14.299278 11866 raft_consensus.cc:374] T b0745260aec04822b3be9ee86f30a507 P ab64a1a3646a40e0baec01867840004c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:14.299297 11866 raft_consensus.cc:729] T b0745260aec04822b3be9ee86f30a507 P ab64a1a3646a40e0baec01867840004c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab64a1a3646a40e0baec01867840004c, State: Initialized, Role: FOLLOWER
I20260926 14:56:14.299376 11866 consensus_queue.cc:260] T b0745260aec04822b3be9ee86f30a507 P ab64a1a3646a40e0baec01867840004c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab64a1a3646a40e0baec01867840004c" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 46169 } } peers { permanent_uuid: "dd55cb52c56c47bab3d2c439d32580a5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38683 } } peers { permanent_uuid: "4a6c5100deae434bbea5b2a0e6b6c00b" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 36345 } }
I20260926 14:56:14.299578 11866 ts_tablet_manager.cc:1428] T b0745260aec04822b3be9ee86f30a507 P ab64a1a3646a40e0baec01867840004c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:56:14.299612 11585 heartbeater.cc:503] Master 127.7.37.254:42929 was elected leader, sending a full tablet report...
W20260926 14:56:14.376128 11719 tablet.cc:2369] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 14:56:14.489192 11586 tablet.cc:2369] T b0745260aec04822b3be9ee86f30a507 P ab64a1a3646a40e0baec01867840004c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:56:14.507501 11872 raft_consensus.cc:482] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:56:14.507593 11872 raft_consensus.cc:504] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab64a1a3646a40e0baec01867840004c" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 46169 } } peers { permanent_uuid: "dd55cb52c56c47bab3d2c439d32580a5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38683 } } peers { permanent_uuid: "4a6c5100deae434bbea5b2a0e6b6c00b" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 36345 } }
I20260926 14:56:14.507915 11872 leader_election.cc:288] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab64a1a3646a40e0baec01867840004c (127.7.37.193:46169), dd55cb52c56c47bab3d2c439d32580a5 (127.7.37.195:38683)
W20260926 14:56:14.509469 11852 tablet.cc:2369] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:56:14.510768 11861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.508138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:40989 (local address 127.7.37.195:38683)
0926 14:56:14.508177 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:14.508179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:14.508186 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:14.508209 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:14.508210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:14.508219 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:14.508265 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:14.508485 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.509086 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.510125 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.510492 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.510551 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:14.510559 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:14.510681 (+ 122us) server_negotiation.cc:299] Negotiation successful
0926 14:56:14.510709 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 14:56:14.510769 11720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.508048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:38683 (local address 127.7.37.194:40989)
0926 14:56:14.508096 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:14.508108 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:14.508157 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:14.508279 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:14.508280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:14.508281 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:14.508329 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:14.508331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.509151 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.509152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:14.510047 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:14.510049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.510555 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.510556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:14.510622 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:14.510626 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:56:14.510670 (+ 44us) client_negotiation.cc:240] Negotiation successful
0926 14:56:14.510704 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 14:56:14.510818 11862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.508134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:56847 (local address 127.7.37.193:46169)
0926 14:56:14.508175 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:14.508176 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:14.508303 (+ 127us) server_negotiation.cc:407] Connection header received
0926 14:56:14.508394 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:14.508395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:14.508406 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:14.508450 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:14.508596 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.509165 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.510171 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.510528 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.510585 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:14.510593 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:14.510727 (+ 134us) server_negotiation.cc:299] Negotiation successful
0926 14:56:14.510761 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:56:14.510860 11874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.508048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:46169 (local address 127.7.37.194:56847)
0926 14:56:14.508230 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:14.508239 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:14.508328 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:14.508463 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:14.508464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:14.508466 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:14.508536 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:14.508538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.509225 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.509226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:14.510098 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:14.510101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.510595 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.510596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:14.510673 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:14.510676 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:14.510717 (+ 41us) client_negotiation.cc:240] Negotiation successful
0926 14:56:14.510753 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":61,"thread_start_us":80,"threads_started":1}
I20260926 14:56:14.511085 11805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0745260aec04822b3be9ee86f30a507" candidate_uuid: "4a6c5100deae434bbea5b2a0e6b6c00b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd55cb52c56c47bab3d2c439d32580a5" is_pre_election: true
I20260926 14:56:14.511117 11539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0745260aec04822b3be9ee86f30a507" candidate_uuid: "4a6c5100deae434bbea5b2a0e6b6c00b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab64a1a3646a40e0baec01867840004c" is_pre_election: true
I20260926 14:56:14.511232 11539 raft_consensus.cc:2454] T b0745260aec04822b3be9ee86f30a507 P ab64a1a3646a40e0baec01867840004c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a6c5100deae434bbea5b2a0e6b6c00b in term 0.
I20260926 14:56:14.511227 11805 raft_consensus.cc:2454] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a6c5100deae434bbea5b2a0e6b6c00b in term 0.
I20260926 14:56:14.511416 11605 leader_election.cc:302] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a6c5100deae434bbea5b2a0e6b6c00b, ab64a1a3646a40e0baec01867840004c; no voters:
I20260926 14:56:14.511543 11872 raft_consensus.cc:2781] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:56:14.511607 11872 raft_consensus.cc:482] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:56:14.511624 11872 raft_consensus.cc:3037] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:14.512235 11872 raft_consensus.cc:504] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab64a1a3646a40e0baec01867840004c" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 46169 } } peers { permanent_uuid: "dd55cb52c56c47bab3d2c439d32580a5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38683 } } peers { permanent_uuid: "4a6c5100deae434bbea5b2a0e6b6c00b" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 36345 } }
I20260926 14:56:14.512336 11872 leader_election.cc:288] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [CANDIDATE]: Term 1 election: Requested vote from peers ab64a1a3646a40e0baec01867840004c (127.7.37.193:46169), dd55cb52c56c47bab3d2c439d32580a5 (127.7.37.195:38683)
I20260926 14:56:14.512461 11805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0745260aec04822b3be9ee86f30a507" candidate_uuid: "4a6c5100deae434bbea5b2a0e6b6c00b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd55cb52c56c47bab3d2c439d32580a5"
I20260926 14:56:14.512502 11539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0745260aec04822b3be9ee86f30a507" candidate_uuid: "4a6c5100deae434bbea5b2a0e6b6c00b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab64a1a3646a40e0baec01867840004c"
I20260926 14:56:14.512542 11805 raft_consensus.cc:3037] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:14.512558 11539 raft_consensus.cc:3037] T b0745260aec04822b3be9ee86f30a507 P ab64a1a3646a40e0baec01867840004c [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:14.513324 11805 raft_consensus.cc:2454] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a6c5100deae434bbea5b2a0e6b6c00b in term 1.
I20260926 14:56:14.513324 11539 raft_consensus.cc:2454] T b0745260aec04822b3be9ee86f30a507 P ab64a1a3646a40e0baec01867840004c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a6c5100deae434bbea5b2a0e6b6c00b in term 1.
I20260926 14:56:14.513497 11605 leader_election.cc:302] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a6c5100deae434bbea5b2a0e6b6c00b, ab64a1a3646a40e0baec01867840004c; no voters:
I20260926 14:56:14.513572 11872 raft_consensus.cc:2781] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:14.513680 11872 raft_consensus.cc:686] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [term 1 LEADER]: Becoming Leader. State: Replica: 4a6c5100deae434bbea5b2a0e6b6c00b, State: Running, Role: LEADER
I20260926 14:56:14.513749 11872 consensus_queue.cc:237] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab64a1a3646a40e0baec01867840004c" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 46169 } } peers { permanent_uuid: "dd55cb52c56c47bab3d2c439d32580a5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38683 } } peers { permanent_uuid: "4a6c5100deae434bbea5b2a0e6b6c00b" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 36345 } }
I20260926 14:56:14.514463 11394 catalog_manager.cc:5495] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a6c5100deae434bbea5b2a0e6b6c00b (127.7.37.194). New cstate: current_term: 1 leader_uuid: "4a6c5100deae434bbea5b2a0e6b6c00b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab64a1a3646a40e0baec01867840004c" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 46169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd55cb52c56c47bab3d2c439d32580a5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a6c5100deae434bbea5b2a0e6b6c00b" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 36345 } health_report { overall_health: HEALTHY } } }
I20260926 14:56:14.568241 11863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.566139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53814 (local address 127.7.37.194:36345)
0926 14:56:14.566187 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:14.566189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:14.566196 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:14.566223 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:14.566224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:14.566241 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:14.566303 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:14.566410 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.566869 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.567564 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:14.567768 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:14.567819 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:14.567947 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:14.568040 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:14.568106 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:14.568137 (+ 31us) server_negotiation.cc:299] Negotiation successful
0926 14:56:14.568155 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 14:56:14.569689 11805 raft_consensus.cc:1261] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5 [term 1 FOLLOWER]: Refusing update from remote peer 4a6c5100deae434bbea5b2a0e6b6c00b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:14.569689 11539 raft_consensus.cc:1261] T b0745260aec04822b3be9ee86f30a507 P ab64a1a3646a40e0baec01867840004c [term 1 FOLLOWER]: Refusing update from remote peer 4a6c5100deae434bbea5b2a0e6b6c00b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:14.570008 11875 consensus_queue.cc:1035] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd55cb52c56c47bab3d2c439d32580a5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:14.570129 11872 consensus_queue.cc:1035] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab64a1a3646a40e0baec01867840004c" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 46169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:14.572336 11879 mvcc.cc:205] Tried to move back new op lower bound from 7333620017435267072 to 7333620017208909824. Current Snapshot: MvccSnapshot[applied={T|T < 7333620017435267072}]
I20260926 14:56:14.574275 11395 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:14.574349 11395 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:14.574925 11395 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 279ca7ba271b4177a95f20f743686394: Sending DeleteTablet for 3 replicas of tablet b0745260aec04822b3be9ee86f30a507
I20260926 14:56:14.575201 11519 tablet_service.cc:1514] Processing DeleteTablet for tablet b0745260aec04822b3be9ee86f30a507 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:14 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:38482
I20260926 14:56:14.575249 11785 tablet_service.cc:1514] Processing DeleteTablet for tablet b0745260aec04822b3be9ee86f30a507 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:14 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:56340
I20260926 14:56:14.575443 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 11457
I20260926 14:56:14.575519 11652 tablet_service.cc:1514] Processing DeleteTablet for tablet b0745260aec04822b3be9ee86f30a507 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:14 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:53802
I20260926 14:56:14.575455 11888 tablet_replica.cc:330] T b0745260aec04822b3be9ee86f30a507 P ab64a1a3646a40e0baec01867840004c: stopping tablet replica
I20260926 14:56:14.575703 11888 raft_consensus.cc:2229] T b0745260aec04822b3be9ee86f30a507 P ab64a1a3646a40e0baec01867840004c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:14.575811 11888 raft_consensus.cc:2258] T b0745260aec04822b3be9ee86f30a507 P ab64a1a3646a40e0baec01867840004c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:14.576156 11889 tablet_replica.cc:330] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5: stopping tablet replica
I20260926 14:56:14.576243 11889 raft_consensus.cc:2229] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:14.576330 11889 raft_consensus.cc:2258] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:14.576686 11889 ts_tablet_manager.cc:1905] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:14.578189 11889 ts_tablet_manager.cc:1918] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:56:14.578253 11889 log.cc:1196] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/b0745260aec04822b3be9ee86f30a507
I20260926 14:56:14.578478 11889 ts_tablet_manager.cc:1939] T b0745260aec04822b3be9ee86f30a507 P dd55cb52c56c47bab3d2c439d32580a5: Deleting consensus metadata
I20260926 14:56:14.578828 11382 catalog_manager.cc:4841] TS dd55cb52c56c47bab3d2c439d32580a5 (127.7.37.195:38683): tablet b0745260aec04822b3be9ee86f30a507 (table test-table [id=af07600220384381a0ba2b8e6a572860]) successfully deleted
W20260926 14:56:14.580606 11379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.37.193:46169 (error 108)
I20260926 14:56:14.581007 11859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.580885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:46169 (local address 127.0.0.1:38488)
0926 14:56:14.580932 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:14.580974 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.193:46169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 14:56:14.581058 11890 tablet_replica.cc:330] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b: stopping tablet replica
I20260926 14:56:14.581053 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 11590
I20260926 14:56:14.581135 11890 raft_consensus.cc:2229] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:56:14.581210 11890 raft_consensus.cc:2258] T b0745260aec04822b3be9ee86f30a507 P 4a6c5100deae434bbea5b2a0e6b6c00b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 14:56:14.583006 11379 catalog_manager.cc:4584] TS ab64a1a3646a40e0baec01867840004c (127.7.37.193:46169): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b0745260aec04822b3be9ee86f30a507: Network error: Client connection negotiation failed: client connection to 127.7.37.193:46169: connect: Connection refused (error 111)
I20260926 14:56:14.585526 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 11723
I20260926 14:56:14.585585 11859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.585489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:36345 (local address 127.0.0.1:53826)
0926 14:56:14.585533 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:14.585559 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.194:36345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 14:56:14.589893 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 11364
2026-09-26T14:56:14Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1936 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-26T14:56:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:14Z Disabled control of system clock
I20260926 14:56:14.613516 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36791
--webserver_interface=127.7.37.254
--webserver_port=0
--builtin_ntp_servers=127.7.37.212:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:36791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 14:56:14.687255 11895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:14.687427 11895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:14.687462 11895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:14.688548 11895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:14.688584 11895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:14.688602 11895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:14.688618 11895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:14.688633 11895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:56:14.689906 11895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:36791
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:14.690112 11895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:14.690338 11895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:14.694558 11901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:14.694586 11900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:14.694586 11903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:14.694757 11895 server_base.cc:1034] running on GCE node
I20260926 14:56:14.694959 11895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:14.695187 11895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:14.696313 11895 hybrid_clock.cc:648] HybridClock initialized: now 1790434574696301 us; error 27 us; skew 500 ppm
I20260926 14:56:14.697340 11895 webserver.cc:460] Webserver started at http://127.7.37.254:35951/ using document root <none> and password file <none>
I20260926 14:56:14.697512 11895 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:14.697549 11895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:14.697629 11895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:14.698498 11895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "2d7a7b584e2d4b0b9e3ea0f1ed399e43"
format_stamp: "Formatted at 2026-09-26 14:56:14 on dist-test-slave-f9xb"
server_key: "87730599686bbc72e153a101de4a566e"
server_key_iv: "9a01e4105267b2f8e8736ea5db57d6d2"
server_key_version: "encryptionkey@0"
I20260926 14:56:14.698750 11895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "2d7a7b584e2d4b0b9e3ea0f1ed399e43"
format_stamp: "Formatted at 2026-09-26 14:56:14 on dist-test-slave-f9xb"
server_key: "87730599686bbc72e153a101de4a566e"
server_key_iv: "9a01e4105267b2f8e8736ea5db57d6d2"
server_key_version: "encryptionkey@0"
I20260926 14:56:14.699831 11895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:14.700444 11909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:14.700575 11895 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:14.700647 11895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "2d7a7b584e2d4b0b9e3ea0f1ed399e43"
format_stamp: "Formatted at 2026-09-26 14:56:14 on dist-test-slave-f9xb"
server_key: "87730599686bbc72e153a101de4a566e"
server_key_iv: "9a01e4105267b2f8e8736ea5db57d6d2"
server_key_version: "encryptionkey@0"
I20260926 14:56:14.700701 11895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:14.703629 11895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:14.703860 11895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:14.703979 11895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:14.707314 11895 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:36791
I20260926 14:56:14.707341 11961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:36791 every 8 connection(s)
I20260926 14:56:14.707593 11895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:56:14.708114 11962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:14.710412 11962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43: Bootstrap starting.
I20260926 14:56:14.710862 11962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:14.711084 11962 log.cc:824] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:14.711611 11962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43: No bootstrap required, opened a new log
I20260926 14:56:14.713024 11962 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d7a7b584e2d4b0b9e3ea0f1ed399e43" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36791 } }
I20260926 14:56:14.713137 11962 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:14.713157 11962 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d7a7b584e2d4b0b9e3ea0f1ed399e43, State: Initialized, Role: FOLLOWER
I20260926 14:56:14.713258 11962 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d7a7b584e2d4b0b9e3ea0f1ed399e43" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36791 } }
I20260926 14:56:14.713325 11962 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:56:14.713352 11962 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:56:14.713382 11962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:14.713798 11962 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d7a7b584e2d4b0b9e3ea0f1ed399e43" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36791 } }
I20260926 14:56:14.713871 11962 leader_election.cc:302] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d7a7b584e2d4b0b9e3ea0f1ed399e43; no voters:
I20260926 14:56:14.713976 11962 leader_election.cc:288] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:56:14.714032 11965 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:14.714170 11965 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43 [term 1 LEADER]: Becoming Leader. State: Replica: 2d7a7b584e2d4b0b9e3ea0f1ed399e43, State: Running, Role: LEADER
I20260926 14:56:14.714172 11962 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:14.714257 11965 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d7a7b584e2d4b0b9e3ea0f1ed399e43" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36791 } }
I20260926 14:56:14.714586 11967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d7a7b584e2d4b0b9e3ea0f1ed399e43. Latest consensus state: current_term: 1 leader_uuid: "2d7a7b584e2d4b0b9e3ea0f1ed399e43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d7a7b584e2d4b0b9e3ea0f1ed399e43" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36791 } } }
I20260926 14:56:14.714663 11967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:14.714841 11966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d7a7b584e2d4b0b9e3ea0f1ed399e43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d7a7b584e2d4b0b9e3ea0f1ed399e43" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36791 } } }
I20260926 14:56:14.714952 11966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:14.715099 11974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:56:14.715463 11974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:56:14.716624 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 11895
I20260926 14:56:14.716713 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:56:14.716888 7319 external_mini_cluster.cc:1428] Setting key ad592fb342419658cb798b2bf4607c44
I20260926 14:56:14.716897 11974 catalog_manager.cc:1347] Generated new cluster ID: 93da7b482a17423881b2fa0f1edf49cf
I20260926 14:56:14.716951 11974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:56:14.721201 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.717260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59982 (local address 127.7.37.254:36791)
0926 14:56:14.717422 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:14.717429 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:14.719931 (+ 2502us) server_negotiation.cc:407] Connection header received
0926 14:56:14.720042 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:14.720057 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:14.720098 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:14.720238 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:14.720798 (+ 560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:14.720802 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:14.720802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:14.720806 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:14.720846 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:14.720860 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:14.720926 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:14.720975 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 14:56:14.721027 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260926 14:56:14.724020 11974 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:56:14.724149 11974 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:14.727801 11974 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43: Generated new TSK 0
I20260926 14:56:14.727957 11974 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:56:14.731508 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36791
--builtin_ntp_servers=127.7.37.212:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 14:56:14.806890 11986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:14.807066 11986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:14.807086 11986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:14.808221 11986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:14.808256 11986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:14.808280 11986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:56:14.809540 11986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:14.809729 11986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:14.809983 11986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 14:56:14.814514 11986 server_base.cc:1034] running on GCE node
W20260926 14:56:14.814538 11992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:14.814618 11991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:14.814553 11994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:14.814895 11986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:14.815114 11986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:14.816269 11986 hybrid_clock.cc:648] HybridClock initialized: now 1790434574816260 us; error 24 us; skew 500 ppm
I20260926 14:56:14.817332 11986 webserver.cc:460] Webserver started at http://127.7.37.193:44549/ using document root <none> and password file <none>
I20260926 14:56:14.817548 11986 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:14.817588 11986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:14.817701 11986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:14.818622 11986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "2da232740bd947f9819d34a3602858f5"
format_stamp: "Formatted at 2026-09-26 14:56:14 on dist-test-slave-f9xb"
server_key: "70e0f02879bbdd02cc38af1533716dc0"
server_key_iv: "937b29282d8b1b75cb134a85f1b3e55a"
server_key_version: "encryptionkey@0"
I20260926 14:56:14.818914 11986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "2da232740bd947f9819d34a3602858f5"
format_stamp: "Formatted at 2026-09-26 14:56:14 on dist-test-slave-f9xb"
server_key: "70e0f02879bbdd02cc38af1533716dc0"
server_key_iv: "937b29282d8b1b75cb134a85f1b3e55a"
server_key_version: "encryptionkey@0"
I20260926 14:56:14.820067 11986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 14:56:14.820703 12000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:14.820849 11986 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:56:14.820916 11986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "2da232740bd947f9819d34a3602858f5"
format_stamp: "Formatted at 2026-09-26 14:56:14 on dist-test-slave-f9xb"
server_key: "70e0f02879bbdd02cc38af1533716dc0"
server_key_iv: "937b29282d8b1b75cb134a85f1b3e55a"
server_key_version: "encryptionkey@0"
I20260926 14:56:14.820986 11986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:14.823961 11986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:14.824180 11986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:14.824291 11986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:14.824461 11986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:14.824741 11986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:14.824779 11986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:14.824805 11986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:14.824827 11986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:14.830782 11986 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:40041
I20260926 14:56:14.830818 12113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:40041 every 8 connection(s)
I20260926 14:56:14.831087 11986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:56:14.833463 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.831651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:50979 (local address 127.7.37.254:36791)
0926 14:56:14.831690 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:14.831692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:14.831946 (+ 254us) server_negotiation.cc:407] Connection header received
0926 14:56:14.832314 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:14.832316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:14.832320 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:14.832354 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:14.833317 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:14.833317 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:14.833318 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:14.833320 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:14.833329 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:14.833332 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:14.833362 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:14.833408 (+ 46us) server_negotiation.cc:299] Negotiation successful
0926 14:56:14.833417 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 14:56:14.833626 12116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.831655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36791 (local address 127.7.37.193:50979)
0926 14:56:14.831840 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:14.831868 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:14.832236 (+ 368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:14.832405 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:14.832408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:14.832415 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:14.833210 (+ 795us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:56:14.833210 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:14.833235 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:56:14.833260 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:14.833367 (+ 107us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:14.833369 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:14.833371 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:56:14.833413 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 14:56:14.833454 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
I20260926 14:56:14.834033 12114 heartbeater.cc:347] Connected to a master server at 127.7.37.254:36791
I20260926 14:56:14.834117 12114 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:14.834239 12114 heartbeater.cc:511] Master 127.7.37.254:36791 requested a full tablet report, sending...
I20260926 14:56:14.834651 11926 ts_manager.cc:194] Registered new tserver with Master: 2da232740bd947f9819d34a3602858f5 (127.7.37.193:40041)
I20260926 14:56:14.835533 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 11986
I20260926 14:56:14.835616 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:56:14.835783 7319 external_mini_cluster.cc:1428] Setting key 5acada025391f728e612853f195b47ea
I20260926 14:56:14.836494 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36791
--builtin_ntp_servers=127.7.37.212:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 14:56:14.910503 12117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:14.910667 12117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:14.910694 12117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:14.911836 12117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:14.911870 12117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:14.911903 12117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:56:14.913157 12117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:14.913363 12117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:14.913605 12117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 14:56:14.918277 12117 server_base.cc:1034] running on GCE node
W20260926 14:56:14.918210 12122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:14.918314 12125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:14.918219 12123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:14.918586 12117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:14.918790 12117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:14.919917 12117 hybrid_clock.cc:648] HybridClock initialized: now 1790434574919909 us; error 24 us; skew 500 ppm
I20260926 14:56:14.920966 12117 webserver.cc:460] Webserver started at http://127.7.37.194:42999/ using document root <none> and password file <none>
I20260926 14:56:14.921166 12117 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:14.921211 12117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:14.921325 12117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:14.922235 12117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "799e5cb5d1ed4cda98a37c31d631bc7c"
format_stamp: "Formatted at 2026-09-26 14:56:14 on dist-test-slave-f9xb"
server_key: "892f2dfa011105ab25a257c1629e5eb5"
server_key_iv: "5488dbc0562782457792feac4cda853e"
server_key_version: "encryptionkey@0"
I20260926 14:56:14.922525 12117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "799e5cb5d1ed4cda98a37c31d631bc7c"
format_stamp: "Formatted at 2026-09-26 14:56:14 on dist-test-slave-f9xb"
server_key: "892f2dfa011105ab25a257c1629e5eb5"
server_key_iv: "5488dbc0562782457792feac4cda853e"
server_key_version: "encryptionkey@0"
I20260926 14:56:14.923768 12117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 14:56:14.924435 12131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:14.924576 12117 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:14.924647 12117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "799e5cb5d1ed4cda98a37c31d631bc7c"
format_stamp: "Formatted at 2026-09-26 14:56:14 on dist-test-slave-f9xb"
server_key: "892f2dfa011105ab25a257c1629e5eb5"
server_key_iv: "5488dbc0562782457792feac4cda853e"
server_key_version: "encryptionkey@0"
I20260926 14:56:14.924708 12117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:14.927635 12117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:14.927874 12117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:14.927982 12117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:14.928145 12117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:14.928383 12117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:14.928413 12117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:14.928436 12117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:14.928459 12117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:14.933566 12117 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:46665
I20260926 14:56:14.933630 12244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:46665 every 8 connection(s)
I20260926 14:56:14.933864 12117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:56:14.936218 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.934401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:56229 (local address 127.7.37.254:36791)
0926 14:56:14.934447 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:14.934449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:14.934628 (+ 179us) server_negotiation.cc:407] Connection header received
0926 14:56:14.934991 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:14.934992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:14.934996 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:14.935042 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:14.936051 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:14.936052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:14.936053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:14.936056 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:14.936070 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:14.936074 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:14.936113 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:14.936158 (+ 45us) server_negotiation.cc:299] Negotiation successful
0926 14:56:14.936169 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 14:56:14.936429 12247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:14.934370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36791 (local address 127.7.37.194:56229)
0926 14:56:14.934528 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:14.934561 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:14.934926 (+ 365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:14.935087 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:14.935090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:14.935098 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:14.935936 (+ 838us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:56:14.935936 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:14.935960 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:56:14.935984 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:14.936117 (+ 133us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:14.936120 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:14.936122 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:56:14.936163 (+ 41us) client_negotiation.cc:240] Negotiation successful
0926 14:56:14.936202 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":79,"threads_started":1}
I20260926 14:56:14.936789 12245 heartbeater.cc:347] Connected to a master server at 127.7.37.254:36791
I20260926 14:56:14.936860 12245 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:14.936975 12245 heartbeater.cc:511] Master 127.7.37.254:36791 requested a full tablet report, sending...
I20260926 14:56:14.937254 11926 ts_manager.cc:194] Registered new tserver with Master: 799e5cb5d1ed4cda98a37c31d631bc7c (127.7.37.194:46665)
I20260926 14:56:14.939900 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 12117
I20260926 14:56:14.939980 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:56:14.940135 7319 external_mini_cluster.cc:1428] Setting key a30507d02b3b2f810f887deb48b4749f
I20260926 14:56:14.940851 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36791
--builtin_ntp_servers=127.7.37.212:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 14:56:15.014564 12248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:15.014747 12248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:15.014775 12248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:15.015921 12248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:15.015954 12248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:15.015977 12248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:56:15.017205 12248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:15.017364 12248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:15.017567 12248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:15.021852 12256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:15.021857 12253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:15.021862 12254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:15.022024 12248 server_base.cc:1034] running on GCE node
I20260926 14:56:15.022244 12248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:15.022439 12248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:15.023609 12248 hybrid_clock.cc:648] HybridClock initialized: now 1790434575023596 us; error 28 us; skew 500 ppm
I20260926 14:56:15.024756 12248 webserver.cc:460] Webserver started at http://127.7.37.195:43151/ using document root <none> and password file <none>
I20260926 14:56:15.024960 12248 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:15.025007 12248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:15.025116 12248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:15.026008 12248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "1fa7da586e5c44dfbd8e1ffaeb2ea9d2"
format_stamp: "Formatted at 2026-09-26 14:56:15 on dist-test-slave-f9xb"
server_key: "5b8e07a0f6023bc74ad5b8f8495ff9bf"
server_key_iv: "f4203699f8213969855d3ce7bda6a9ef"
server_key_version: "encryptionkey@0"
I20260926 14:56:15.026296 12248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "1fa7da586e5c44dfbd8e1ffaeb2ea9d2"
format_stamp: "Formatted at 2026-09-26 14:56:15 on dist-test-slave-f9xb"
server_key: "5b8e07a0f6023bc74ad5b8f8495ff9bf"
server_key_iv: "f4203699f8213969855d3ce7bda6a9ef"
server_key_version: "encryptionkey@0"
I20260926 14:56:15.027446 12248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 14:56:15.028297 12262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:15.028476 12248 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:15.028561 12248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "1fa7da586e5c44dfbd8e1ffaeb2ea9d2"
format_stamp: "Formatted at 2026-09-26 14:56:15 on dist-test-slave-f9xb"
server_key: "5b8e07a0f6023bc74ad5b8f8495ff9bf"
server_key_iv: "f4203699f8213969855d3ce7bda6a9ef"
server_key_version: "encryptionkey@0"
I20260926 14:56:15.028625 12248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:15.031853 12248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:15.032076 12248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:15.032186 12248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:15.032352 12248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:15.032598 12248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:15.032629 12248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:15.032652 12248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:15.032676 12248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:15.037743 12248 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:36949
I20260926 14:56:15.037776 12375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:36949 every 8 connection(s)
I20260926 14:56:15.038030 12248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:56:15.040387 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.038602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:58019 (local address 127.7.37.254:36791)
0926 14:56:15.038654 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:15.038656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:15.038831 (+ 175us) server_negotiation.cc:407] Connection header received
0926 14:56:15.039211 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:15.039212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:15.039218 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:15.039257 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:15.040213 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:15.040214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:15.040214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:15.040218 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:15.040232 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:15.040235 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:15.040277 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:15.040325 (+ 48us) server_negotiation.cc:299] Negotiation successful
0926 14:56:15.040338 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 14:56:15.040532 12378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.038578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36791 (local address 127.7.37.195:58019)
0926 14:56:15.038732 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:15.038765 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:15.039130 (+ 365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:15.039308 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:15.039311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:15.039318 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:15.040097 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:56:15.040097 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:15.040121 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:56:15.040145 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:15.040275 (+ 130us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:15.040277 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:15.040279 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:56:15.040324 (+ 45us) client_negotiation.cc:240] Negotiation successful
0926 14:56:15.040364 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":75,"threads_started":1}
I20260926 14:56:15.040913 12376 heartbeater.cc:347] Connected to a master server at 127.7.37.254:36791
I20260926 14:56:15.040998 12376 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:15.041108 12376 heartbeater.cc:511] Master 127.7.37.254:36791 requested a full tablet report, sending...
I20260926 14:56:15.041440 11926 ts_manager.cc:194] Registered new tserver with Master: 1fa7da586e5c44dfbd8e1ffaeb2ea9d2 (127.7.37.195:36949)
I20260926 14:56:15.044373 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 12248
I20260926 14:56:15.044457 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:56:15.044615 7319 external_mini_cluster.cc:1428] Setting key 71a42d8adc2811ed60ff92d26375d395
I20260926 14:56:15.045408 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 14:56:15.050058 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.046859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59984 (local address 127.7.37.254:36791)
0926 14:56:15.046905 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:15.046906 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:15.049182 (+ 2276us) server_negotiation.cc:407] Connection header received
0926 14:56:15.049204 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:15.049205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:15.049210 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:15.049250 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:15.049811 (+ 561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:15.049812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:15.049813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:15.049817 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:15.049844 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:15.049848 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:15.049891 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:15.049982 (+ 91us) server_negotiation.cc:299] Negotiation successful
0926 14:56:15.049996 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 14:56:15.052412 11926 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:56:15.054136 11926 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:56:15.057307 12384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.055825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:36949 (local address 127.0.0.1:36162)
0926 14:56:15.056064 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:15.056074 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:15.056123 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:15.056336 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:15.056341 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:15.056346 (+ 5us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:15.056976 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:56:15.056976 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:15.056987 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:56:15.057002 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:15.057168 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:15.057172 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:15.057175 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:15.057229 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 14:56:15.057241 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":1,"spinlock_wait_cycles":384,"thread_start_us":99,"threads_started":1}
I20260926 14:56:15.057315 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.056079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36162 (local address 127.7.37.195:36949)
0926 14:56:15.056231 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:15.056235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:15.056243 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:15.056261 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:15.056264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:15.056272 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:15.056337 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:15.057062 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:15.057067 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:15.057068 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:15.057071 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:15.057100 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:15.057118 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:15.057164 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:15.057232 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 14:56:15.057243 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":65,"threads_started":1}
I20260926 14:56:15.057519 12387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.055907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58010 (local address 127.7.37.193:40041)
0926 14:56:15.056235 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:15.056238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:15.056246 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:15.056269 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:15.056271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:15.056280 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:15.056553 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:15.057202 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:15.057211 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:15.057211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:15.057215 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:15.057246 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:15.057271 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:15.057325 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:15.057419 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 14:56:15.057433 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":73,"threads_started":1}
I20260926 14:56:15.057453 12386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.055825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:40041 (local address 127.0.0.1:58010)
0926 14:56:15.055987 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:15.056020 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:15.056070 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:15.056376 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:15.056377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:15.056379 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:15.057105 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:56:15.057106 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:15.057112 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:56:15.057124 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:15.057331 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:15.057332 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:15.057333 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 14:56:15.057381 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 14:56:15.057394 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":46,"threads_started":1}
I20260926 14:56:15.057948 12385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.055841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:46665 (local address 127.0.0.1:46352)
0926 14:56:15.056195 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:15.056204 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:15.056244 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:15.056647 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:15.056648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:15.056650 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:15.057579 (+ 929us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:56:15.057579 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:15.057586 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:56:15.057597 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:15.057821 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:15.057821 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:15.057822 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 14:56:15.057863 (+ 41us) client_negotiation.cc:240] Negotiation successful
0926 14:56:15.057884 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":93,"threads_started":1}
I20260926 14:56:15.057963 12388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.056056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46352 (local address 127.7.37.194:46665)
0926 14:56:15.056196 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:15.056200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:15.056243 (+ 43us) server_negotiation.cc:407] Connection header received
0926 14:56:15.056288 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:15.056290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:15.056298 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:15.056359 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:15.057672 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:15.057678 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:15.057678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:15.057683 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:15.057717 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:15.057748 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:15.057798 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:15.057872 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 14:56:15.057887 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260926 14:56:15.058290 12310 tablet_service.cc:1467] Processing CreateTablet for tablet eb29107c45a745e78f9b5e45c3ddc311 (DEFAULT_TABLE table=test-table [id=830eb601656f49ceb318b5fcd8b737bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:15.058290 12048 tablet_service.cc:1467] Processing CreateTablet for tablet eb29107c45a745e78f9b5e45c3ddc311 (DEFAULT_TABLE table=test-table [id=830eb601656f49ceb318b5fcd8b737bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:15.058583 12310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb29107c45a745e78f9b5e45c3ddc311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:15.058583 12048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb29107c45a745e78f9b5e45c3ddc311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:15.058493 12179 tablet_service.cc:1467] Processing CreateTablet for tablet eb29107c45a745e78f9b5e45c3ddc311 (DEFAULT_TABLE table=test-table [id=830eb601656f49ceb318b5fcd8b737bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:15.058815 12179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb29107c45a745e78f9b5e45c3ddc311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:15.061416 12391 tablet_bootstrap.cc:492] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2: Bootstrap starting.
I20260926 14:56:15.061856 12390 tablet_bootstrap.cc:492] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c: Bootstrap starting.
I20260926 14:56:15.061950 12391 tablet_bootstrap.cc:654] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:15.062202 12391 log.cc:824] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:15.062354 12390 tablet_bootstrap.cc:654] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:15.062466 12392 tablet_bootstrap.cc:492] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5: Bootstrap starting.
I20260926 14:56:15.062630 12390 log.cc:824] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:15.062829 12391 tablet_bootstrap.cc:492] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2: No bootstrap required, opened a new log
I20260926 14:56:15.062892 12391 ts_tablet_manager.cc:1397] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:56:15.063194 12390 tablet_bootstrap.cc:492] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c: No bootstrap required, opened a new log
I20260926 14:56:15.063241 12390 ts_tablet_manager.cc:1397] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:15.064282 12392 tablet_bootstrap.cc:654] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:15.064586 12392 log.cc:824] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:15.064857 12391 raft_consensus.cc:348] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa7da586e5c44dfbd8e1ffaeb2ea9d2" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36949 } } peers { permanent_uuid: "2da232740bd947f9819d34a3602858f5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40041 } } peers { permanent_uuid: "799e5cb5d1ed4cda98a37c31d631bc7c" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 46665 } }
I20260926 14:56:15.065016 12391 raft_consensus.cc:374] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:15.065056 12391 raft_consensus.cc:729] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fa7da586e5c44dfbd8e1ffaeb2ea9d2, State: Initialized, Role: FOLLOWER
I20260926 14:56:15.065157 12391 consensus_queue.cc:260] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa7da586e5c44dfbd8e1ffaeb2ea9d2" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36949 } } peers { permanent_uuid: "2da232740bd947f9819d34a3602858f5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40041 } } peers { permanent_uuid: "799e5cb5d1ed4cda98a37c31d631bc7c" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 46665 } }
I20260926 14:56:15.065299 12392 tablet_bootstrap.cc:492] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5: No bootstrap required, opened a new log
I20260926 14:56:15.065361 12392 ts_tablet_manager.cc:1397] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 14:56:15.065379 12391 ts_tablet_manager.cc:1428] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260926 14:56:15.065477 12376 heartbeater.cc:503] Master 127.7.37.254:36791 was elected leader, sending a full tablet report...
I20260926 14:56:15.065418 12390 raft_consensus.cc:348] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa7da586e5c44dfbd8e1ffaeb2ea9d2" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36949 } } peers { permanent_uuid: "2da232740bd947f9819d34a3602858f5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40041 } } peers { permanent_uuid: "799e5cb5d1ed4cda98a37c31d631bc7c" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 46665 } }
I20260926 14:56:15.065560 12390 raft_consensus.cc:374] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:15.065609 12390 raft_consensus.cc:729] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 799e5cb5d1ed4cda98a37c31d631bc7c, State: Initialized, Role: FOLLOWER
I20260926 14:56:15.065712 12390 consensus_queue.cc:260] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa7da586e5c44dfbd8e1ffaeb2ea9d2" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36949 } } peers { permanent_uuid: "2da232740bd947f9819d34a3602858f5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40041 } } peers { permanent_uuid: "799e5cb5d1ed4cda98a37c31d631bc7c" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 46665 } }
I20260926 14:56:15.065945 12390 ts_tablet_manager.cc:1428] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 14:56:15.065965 12245 heartbeater.cc:503] Master 127.7.37.254:36791 was elected leader, sending a full tablet report...
I20260926 14:56:15.067468 12392 raft_consensus.cc:348] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa7da586e5c44dfbd8e1ffaeb2ea9d2" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36949 } } peers { permanent_uuid: "2da232740bd947f9819d34a3602858f5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40041 } } peers { permanent_uuid: "799e5cb5d1ed4cda98a37c31d631bc7c" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 46665 } }
I20260926 14:56:15.067584 12392 raft_consensus.cc:374] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:15.067601 12392 raft_consensus.cc:729] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2da232740bd947f9819d34a3602858f5, State: Initialized, Role: FOLLOWER
I20260926 14:56:15.067698 12392 consensus_queue.cc:260] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa7da586e5c44dfbd8e1ffaeb2ea9d2" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36949 } } peers { permanent_uuid: "2da232740bd947f9819d34a3602858f5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40041 } } peers { permanent_uuid: "799e5cb5d1ed4cda98a37c31d631bc7c" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 46665 } }
I20260926 14:56:15.067929 12392 ts_tablet_manager.cc:1428] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 14:56:15.067958 12114 heartbeater.cc:503] Master 127.7.37.254:36791 was elected leader, sending a full tablet report...
W20260926 14:56:15.081527 12115 tablet.cc:2369] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 14:56:15.184337 12246 tablet.cc:2369] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:56:15.266373 12398 raft_consensus.cc:482] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:56:15.266485 12398 raft_consensus.cc:504] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa7da586e5c44dfbd8e1ffaeb2ea9d2" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36949 } } peers { permanent_uuid: "2da232740bd947f9819d34a3602858f5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40041 } } peers { permanent_uuid: "799e5cb5d1ed4cda98a37c31d631bc7c" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 46665 } }
I20260926 14:56:15.266760 12398 leader_election.cc:288] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fa7da586e5c44dfbd8e1ffaeb2ea9d2 (127.7.37.195:36949), 799e5cb5d1ed4cda98a37c31d631bc7c (127.7.37.194:46665)
I20260926 14:56:15.267863 12116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.266914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:36949 (local address 127.7.37.193:57275)
0926 14:56:15.266951 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:15.266961 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:15.266996 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:15.267129 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:15.267130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:15.267132 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:15.267560 (+ 428us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:56:15.267560 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:15.267568 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:56:15.267581 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:15.267758 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:15.267759 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:15.267759 (+ 0us) client_negotiation.cc:769] Sending connection context
0926 14:56:15.267805 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 14:56:15.267816 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 14:56:15.267985 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.266975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:57275 (local address 127.7.37.195:36949)
0926 14:56:15.267014 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:15.267016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:15.267026 (+ 10us) server_negotiation.cc:407] Connection header received
0926 14:56:15.267053 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:15.267054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:15.267060 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:15.267110 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:15.267655 (+ 545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:15.267656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:15.267656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:15.267660 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:15.267678 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:15.267683 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:15.267761 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:15.267902 (+ 141us) server_negotiation.cc:299] Negotiation successful
0926 14:56:15.267915 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 14:56:15.268057 12388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.266986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:49655 (local address 127.7.37.194:46665)
0926 14:56:15.267034 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:15.267036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:15.267237 (+ 201us) server_negotiation.cc:407] Connection header received
0926 14:56:15.267296 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:15.267297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:15.267302 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:15.267347 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:15.267811 (+ 464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:15.267812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:15.267812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:15.267817 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:15.267829 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:15.267832 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:15.267878 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:15.267986 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 14:56:15.268000 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 14:56:15.268079 12399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.266919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:46665 (local address 127.7.37.193:49655)
0926 14:56:15.267179 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:15.267188 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:15.267239 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:15.267354 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:15.267355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:15.267357 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:15.267717 (+ 360us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:56:15.267718 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:15.267725 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:56:15.267735 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:15.267874 (+ 139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:15.267875 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:15.267875 (+ 0us) client_negotiation.cc:769] Sending connection context
0926 14:56:15.267926 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 14:56:15.267938 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":120,"thread_start_us":78,"threads_started":1}
I20260926 14:56:15.268342 12330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb29107c45a745e78f9b5e45c3ddc311" candidate_uuid: "2da232740bd947f9819d34a3602858f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fa7da586e5c44dfbd8e1ffaeb2ea9d2" is_pre_election: true
I20260926 14:56:15.268404 12199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb29107c45a745e78f9b5e45c3ddc311" candidate_uuid: "2da232740bd947f9819d34a3602858f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "799e5cb5d1ed4cda98a37c31d631bc7c" is_pre_election: true
I20260926 14:56:15.268492 12330 raft_consensus.cc:2454] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2da232740bd947f9819d34a3602858f5 in term 0.
I20260926 14:56:15.268550 12199 raft_consensus.cc:2454] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2da232740bd947f9819d34a3602858f5 in term 0.
I20260926 14:56:15.268716 12004 leader_election.cc:302] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fa7da586e5c44dfbd8e1ffaeb2ea9d2, 2da232740bd947f9819d34a3602858f5; no voters:
I20260926 14:56:15.268872 12398 raft_consensus.cc:2781] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:56:15.268929 12398 raft_consensus.cc:482] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:56:15.268945 12398 raft_consensus.cc:3037] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:15.269565 12398 raft_consensus.cc:504] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa7da586e5c44dfbd8e1ffaeb2ea9d2" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36949 } } peers { permanent_uuid: "2da232740bd947f9819d34a3602858f5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40041 } } peers { permanent_uuid: "799e5cb5d1ed4cda98a37c31d631bc7c" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 46665 } }
I20260926 14:56:15.269670 12398 leader_election.cc:288] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 [CANDIDATE]: Term 1 election: Requested vote from peers 1fa7da586e5c44dfbd8e1ffaeb2ea9d2 (127.7.37.195:36949), 799e5cb5d1ed4cda98a37c31d631bc7c (127.7.37.194:46665)
I20260926 14:56:15.269798 12330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb29107c45a745e78f9b5e45c3ddc311" candidate_uuid: "2da232740bd947f9819d34a3602858f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fa7da586e5c44dfbd8e1ffaeb2ea9d2"
I20260926 14:56:15.269871 12330 raft_consensus.cc:3037] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:15.270507 12330 raft_consensus.cc:2454] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2da232740bd947f9819d34a3602858f5 in term 1.
I20260926 14:56:15.270591 12199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb29107c45a745e78f9b5e45c3ddc311" candidate_uuid: "2da232740bd947f9819d34a3602858f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "799e5cb5d1ed4cda98a37c31d631bc7c"
I20260926 14:56:15.270648 12004 leader_election.cc:302] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fa7da586e5c44dfbd8e1ffaeb2ea9d2, 2da232740bd947f9819d34a3602858f5; no voters:
I20260926 14:56:15.270655 12199 raft_consensus.cc:3037] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:15.270745 12398 raft_consensus.cc:2781] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:15.270881 12398 raft_consensus.cc:686] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 [term 1 LEADER]: Becoming Leader. State: Replica: 2da232740bd947f9819d34a3602858f5, State: Running, Role: LEADER
I20260926 14:56:15.270968 12398 consensus_queue.cc:237] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa7da586e5c44dfbd8e1ffaeb2ea9d2" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36949 } } peers { permanent_uuid: "2da232740bd947f9819d34a3602858f5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40041 } } peers { permanent_uuid: "799e5cb5d1ed4cda98a37c31d631bc7c" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 46665 } }
I20260926 14:56:15.271396 12199 raft_consensus.cc:2454] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2da232740bd947f9819d34a3602858f5 in term 1.
I20260926 14:56:15.271885 11925 catalog_manager.cc:5495] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2da232740bd947f9819d34a3602858f5 (127.7.37.193). New cstate: current_term: 1 leader_uuid: "2da232740bd947f9819d34a3602858f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa7da586e5c44dfbd8e1ffaeb2ea9d2" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36949 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2da232740bd947f9819d34a3602858f5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "799e5cb5d1ed4cda98a37c31d631bc7c" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 46665 } health_report { overall_health: UNKNOWN } } }
I20260926 14:56:15.281033 12387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.280314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58024 (local address 127.7.37.193:40041)
0926 14:56:15.280360 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:15.280362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:15.280371 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:56:15.280399 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:15.280400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:15.280410 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:15.280455 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:15.280875 (+ 420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:15.280876 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:15.280877 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:15.280880 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:15.280892 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:15.280895 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:15.280939 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:15.280968 (+ 29us) server_negotiation.cc:299] Negotiation successful
0926 14:56:15.280979 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 14:56:15.282428 12199 raft_consensus.cc:1261] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c [term 1 FOLLOWER]: Refusing update from remote peer 2da232740bd947f9819d34a3602858f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:15.282425 12330 raft_consensus.cc:1261] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2 [term 1 FOLLOWER]: Refusing update from remote peer 2da232740bd947f9819d34a3602858f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:15.282702 12398 consensus_queue.cc:1035] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fa7da586e5c44dfbd8e1ffaeb2ea9d2" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:15.282819 12400 consensus_queue.cc:1035] T eb29107c45a745e78f9b5e45c3ddc311 P 2da232740bd947f9819d34a3602858f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "799e5cb5d1ed4cda98a37c31d631bc7c" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 46665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:15.285308 12403 mvcc.cc:205] Tried to move back new op lower bound from 7333620020354760704 to 7333620020310495232. Current Snapshot: MvccSnapshot[applied={T|T < 7333620020354760704}]
I20260926 14:56:15.287328 11926 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:15.287411 11926 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:15.288025 11926 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d7a7b584e2d4b0b9e3ea0f1ed399e43: Sending DeleteTablet for 3 replicas of tablet eb29107c45a745e78f9b5e45c3ddc311
I20260926 14:56:15.288285 12310 tablet_service.cc:1514] Processing DeleteTablet for tablet eb29107c45a745e78f9b5e45c3ddc311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:15 UTC) from {username='slave'} at 127.0.0.1:36162
I20260926 14:56:15.288324 12179 tablet_service.cc:1514] Processing DeleteTablet for tablet eb29107c45a745e78f9b5e45c3ddc311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:15 UTC) from {username='slave'} at 127.0.0.1:46352
I20260926 14:56:15.288400 12048 tablet_service.cc:1514] Processing DeleteTablet for tablet eb29107c45a745e78f9b5e45c3ddc311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:15 UTC) from {username='slave'} at 127.0.0.1:58010
I20260926 14:56:15.288497 12414 tablet_replica.cc:330] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c: stopping tablet replica
I20260926 14:56:15.288669 12413 tablet_replica.cc:330] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2: stopping tablet replica
I20260926 14:56:15.288727 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 11986
I20260926 14:56:15.288825 12413 raft_consensus.cc:2229] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:15.288959 12413 raft_consensus.cc:2258] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:15.289916 12413 ts_tablet_manager.cc:1905] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:15.291173 12413 ts_tablet_manager.cc:1918] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:56:15.291224 12413 log.cc:1196] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/eb29107c45a745e78f9b5e45c3ddc311
I20260926 14:56:15.291419 12413 ts_tablet_manager.cc:1939] T eb29107c45a745e78f9b5e45c3ddc311 P 1fa7da586e5c44dfbd8e1ffaeb2ea9d2: Deleting consensus metadata
I20260926 14:56:15.291738 11913 catalog_manager.cc:4841] TS 1fa7da586e5c44dfbd8e1ffaeb2ea9d2 (127.7.37.195:36949): tablet eb29107c45a745e78f9b5e45c3ddc311 (table test-table [id=830eb601656f49ceb318b5fcd8b737bb]) successfully deleted
I20260926 14:56:15.292249 12414 raft_consensus.cc:2229] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:15.292356 12414 raft_consensus.cc:2258] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:15.292683 12414 ts_tablet_manager.cc:1905] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 14:56:15.293757 11910 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.37.193:40041 (error 108)
I20260926 14:56:15.294040 12414 ts_tablet_manager.cc:1918] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:56:15.294102 12414 log.cc:1196] T eb29107c45a745e78f9b5e45c3ddc311 P 799e5cb5d1ed4cda98a37c31d631bc7c: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/eb29107c45a745e78f9b5e45c3ddc311
I20260926 14:56:15.294127 12385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.294026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:40041 (local address 127.0.0.1:58036)
0926 14:56:15.294064 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:15.294099 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.193:40041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260926 14:56:15.294142 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 12117
W20260926 14:56:15.294236 11910 catalog_manager.cc:4584] TS 2da232740bd947f9819d34a3602858f5 (127.7.37.193:40041): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eb29107c45a745e78f9b5e45c3ddc311: Network error: Client connection negotiation failed: client connection to 127.7.37.193:40041: connect: Connection refused (error 111)
I20260926 14:56:15.298054 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 12248
I20260926 14:56:15.298091 12385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.298000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:46665 (local address 127.0.0.1:46360)
0926 14:56:15.298040 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:15.298068 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.194:46665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260926 14:56:15.301578 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 11895
2026-09-26T14:56:15Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (711 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260926 14:56:15.319007 7319 mini_oidc.cc:152] Starting JWKS server
I20260926 14:56:15.319959 7319 webserver.cc:460] Webserver started at https://127.0.0.1:41059/ using document root <none> and password file <none>
I20260926 14:56:15.320035 7319 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 14:56:15.320183 7319 webserver.cc:460] Webserver started at http://127.0.0.1:39261/ using document root <none> and password file <none>
2026-09-26T14:56:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:15Z Disabled control of system clock
I20260926 14:56:15.327111 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:33541
--webserver_interface=127.7.37.254
--webserver_port=0
--builtin_ntp_servers=127.7.37.212:37989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:33541
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41059/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0/testchainca.cert with env {}
W20260926 14:56:15.404294 12422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:15.404462 12422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:15.404482 12422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:15.405550 12422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:15.405581 12422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:15.405592 12422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:15.405603 12422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:15.405620 12422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 14:56:15.405632 12422 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 14:56:15.405643 12422 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41059/jwks.json
I20260926 14:56:15.407013 12422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:37989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:33541
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:33541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41059/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:15.407173 12422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:15.407305 12422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:15.411599 12427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:15.411671 12428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:15.411614 12430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:15.411679 12422 server_base.cc:1034] running on GCE node
I20260926 14:56:15.412014 12422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:15.412299 12422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:15.413517 12422 hybrid_clock.cc:648] HybridClock initialized: now 1790434575413503 us; error 25 us; skew 500 ppm
I20260926 14:56:15.419087 12422 webserver.cc:460] Webserver started at http://127.7.37.254:34957/ using document root <none> and password file <none>
I20260926 14:56:15.419271 12422 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:15.419315 12422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:15.419430 12422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:15.420369 12422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "d21d9f0815db40a8b7567985c74e7f45"
format_stamp: "Formatted at 2026-09-26 14:56:15 on dist-test-slave-f9xb"
server_key: "325cc7bceab8baf2103bf998b46c20af"
server_key_iv: "dea36fc843da4bf107521a2f675542cf"
server_key_version: "encryptionkey@0"
I20260926 14:56:15.420660 12422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "d21d9f0815db40a8b7567985c74e7f45"
format_stamp: "Formatted at 2026-09-26 14:56:15 on dist-test-slave-f9xb"
server_key: "325cc7bceab8baf2103bf998b46c20af"
server_key_iv: "dea36fc843da4bf107521a2f675542cf"
server_key_version: "encryptionkey@0"
I20260926 14:56:15.421749 12422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 14:56:15.422394 12436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:15.422540 12422 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:15.422613 12422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "d21d9f0815db40a8b7567985c74e7f45"
format_stamp: "Formatted at 2026-09-26 14:56:15 on dist-test-slave-f9xb"
server_key: "325cc7bceab8baf2103bf998b46c20af"
server_key_iv: "dea36fc843da4bf107521a2f675542cf"
server_key_version: "encryptionkey@0"
I20260926 14:56:15.422667 12422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:15.438136 12422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:15.438345 12422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:15.438454 12422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:15.441824 12422 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:33541
I20260926 14:56:15.441850 12488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:33541 every 8 connection(s)
I20260926 14:56:15.442111 12422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:56:15.442876 12489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:15.444890 12489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45: Bootstrap starting.
I20260926 14:56:15.445391 12489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:15.445631 12489 log.cc:824] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:15.446171 12489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45: No bootstrap required, opened a new log
I20260926 14:56:15.447621 12489 raft_consensus.cc:348] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d21d9f0815db40a8b7567985c74e7f45" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 33541 } }
I20260926 14:56:15.447738 12489 raft_consensus.cc:374] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:15.447806 12489 raft_consensus.cc:729] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d21d9f0815db40a8b7567985c74e7f45, State: Initialized, Role: FOLLOWER
I20260926 14:56:15.447896 12489 consensus_queue.cc:260] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d21d9f0815db40a8b7567985c74e7f45" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 33541 } }
I20260926 14:56:15.447958 12489 raft_consensus.cc:388] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:56:15.447976 12489 raft_consensus.cc:482] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:56:15.448005 12489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:15.448397 12489 raft_consensus.cc:504] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d21d9f0815db40a8b7567985c74e7f45" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 33541 } }
I20260926 14:56:15.448468 12489 leader_election.cc:302] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d21d9f0815db40a8b7567985c74e7f45; no voters:
I20260926 14:56:15.448573 12489 leader_election.cc:288] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:56:15.448614 12492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:15.448730 12492 raft_consensus.cc:686] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45 [term 1 LEADER]: Becoming Leader. State: Replica: d21d9f0815db40a8b7567985c74e7f45, State: Running, Role: LEADER
I20260926 14:56:15.448771 12489 sys_catalog.cc:564] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:15.448822 12492 consensus_queue.cc:237] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d21d9f0815db40a8b7567985c74e7f45" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 33541 } }
I20260926 14:56:15.449159 12494 sys_catalog.cc:455] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d21d9f0815db40a8b7567985c74e7f45. Latest consensus state: current_term: 1 leader_uuid: "d21d9f0815db40a8b7567985c74e7f45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d21d9f0815db40a8b7567985c74e7f45" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 33541 } } }
I20260926 14:56:15.449231 12494 sys_catalog.cc:458] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:15.449321 12493 sys_catalog.cc:455] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d21d9f0815db40a8b7567985c74e7f45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d21d9f0815db40a8b7567985c74e7f45" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 33541 } } }
I20260926 14:56:15.449455 12493 sys_catalog.cc:458] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:15.449635 12500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:56:15.450052 12500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:56:15.450565 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 12422
I20260926 14:56:15.450661 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:56:15.450826 7319 external_mini_cluster.cc:1428] Setting key 1876ed96c09290d83a11d3b29e460a85
I20260926 14:56:15.451385 12500 catalog_manager.cc:1347] Generated new cluster ID: 994a22e4556149eabac722e861afa5d3
I20260926 14:56:15.451429 12500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:56:15.456393 12512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.451179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34756 (local address 127.7.37.254:33541)
0926 14:56:15.451549 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:15.451557 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:15.451585 (+ 28us) server_negotiation.cc:407] Connection header received
0926 14:56:15.451679 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:15.451685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:15.451764 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:15.451900 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:15.452103 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:15.453383 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:15.454446 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:15.455019 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:15.455115 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:15.455694 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:15.455698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:15.455698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:15.455891 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:15.455929 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:15.455942 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:15.455996 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:15.456078 (+ 82us) server_negotiation.cc:299] Negotiation successful
0926 14:56:15.456215 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":310,"thread_start_us":60,"threads_started":1}
I20260926 14:56:15.457110 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:56:15.482432 12500 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:56:15.482963 12500 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:15.494593 12500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d21d9f0815db40a8b7567985c74e7f45: Generated new TSK 0
I20260926 14:56:15.494730 12500 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 14:56:15.523092 12519 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260926 14:56:15.523175 12519 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 14:56:15.523335 12512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.509400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34764 (local address 127.7.37.254:33541)
0926 14:56:15.509460 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:15.509462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:15.509538 (+ 76us) server_negotiation.cc:407] Connection header received
0926 14:56:15.509614 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:15.509614 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:15.509625 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:15.509673 (+ 48us) server_negotiation.cc:226] Negotiated authn=JWT
0926 14:56:15.509792 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:15.510230 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:15.510948 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:15.511139 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:15.511189 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:15.511358 (+ 169us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 14:56:15.523175 (+ 11817us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41059/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 14:56:15.523294 (+ 119us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34764: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41059/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":29}
W20260926 14:56:15.523371 12512 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34764: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41059/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260926 14:56:15.523716 12518 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:15.509335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:33541 (local address 127.0.0.1:34764)
0926 14:56:15.509479 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:15.509488 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:15.509546 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:15.509686 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:15.509687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:15.509688 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0926 14:56:15.509743 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:15.509745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:15.510313 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:15.510314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:15.510880 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:15.510882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:15.511194 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:15.511195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:15.511259 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:15.511275 (+ 16us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 14:56:15.523314 (+ 12039us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41059/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 14:56:15.523434 (+ 120us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.37.254:33541: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41059/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":66,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.37.254:33541: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41059/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260926 14:56:15.524053 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 12422
2026-09-26T14:56:15Z chronyd exiting
I20260926 14:56:15.558688 7319 test_util.cc:175] -----------------------------------------------
I20260926 14:56:15.558743 7319 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434555014799-7319-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (240 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260926 14:56:15.559401 7319 mini_oidc.cc:152] Starting JWKS server
I20260926 14:56:15.560046 7319 webserver.cc:460] Webserver started at https://127.0.0.1:46307/ using document root <none> and password file <none>
I20260926 14:56:15.560086 7319 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 14:56:15.560226 7319 webserver.cc:460] Webserver started at http://127.0.0.1:34473/ using document root <none> and password file <none>
2026-09-26T14:56:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:15Z Disabled control of system clock
I20260926 14:56:15.566342 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:42695
--webserver_interface=127.7.37.254
--webserver_port=0
--builtin_ntp_servers=127.7.37.212:35899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:42695
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46307/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434555014799-7319-0/testchainca.cert with env {}
W20260926 14:56:15.639909 12525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:15.640089 12525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:15.640116 12525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:15.641233 12525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:15.641270 12525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:15.641286 12525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:15.641301 12525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:15.641316 12525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 14:56:15.641340 12525 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 14:56:15.641358 12525 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46307/jwks.json
I20260926 14:56:15.642740 12525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:35899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:42695
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:42695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46307/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434555014799-7319-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:15.642946 12525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:15.643126 12525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:15.647269 12531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:15.647301 12530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:15.647298 12533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:15.647454 12525 server_base.cc:1034] running on GCE node
I20260926 14:56:15.647650 12525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:15.647928 12525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:15.649091 12525 hybrid_clock.cc:648] HybridClock initialized: now 1790434575649073 us; error 28 us; skew 500 ppm
I20260926 14:56:15.650151 12525 webserver.cc:460] Webserver started at http://127.7.37.254:42869/ using document root <none> and password file <none>
I20260926 14:56:15.650344 12525 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:15.650388 12525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:15.650503 12525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:15.651422 12525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "690dd01d5c8f4afca537afccac522c24"
format_stamp: "Formatted at 2026-09-26 14:56:15 on dist-test-slave-f9xb"
server_key: "6ebe717ae8323e39842677ebea1f92d5"
server_key_iv: "6491f863e3ea7e92e8cbdcf57ae91b0f"
server_key_version: "encryptionkey@0"
I20260926 14:56:15.651723 12525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "690dd01d5c8f4afca537afccac522c24"
format_stamp: "Formatted at 2026-09-26 14:56:15 on dist-test-slave-f9xb"
server_key: "6ebe717ae8323e39842677ebea1f92d5"
server_key_iv: "6491f863e3ea7e92e8cbdcf57ae91b0f"
server_key_version: "encryptionkey@0"
I20260926 14:56:15.652845 12525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:15.653416 12539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:15.653548 12525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:15.653621 12525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "690dd01d5c8f4afca537afccac522c24"
format_stamp: "Formatted at 2026-09-26 14:56:15 on dist-test-slave-f9xb"
server_key: "6ebe717ae8323e39842677ebea1f92d5"
server_key_iv: "6491f863e3ea7e92e8cbdcf57ae91b0f"
server_key_version: "encryptionkey@0"
I20260926 14:56:15.653682 12525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:15.678676 12525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:15.678898 12525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:15.679018 12525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:15.682426 12525 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:42695
I20260926 14:56:15.682456 12591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:42695 every 8 connection(s)
I20260926 14:56:15.682703 12525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:56:15.683473 12592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:15.685345 12592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24: Bootstrap starting.
I20260926 14:56:15.685802 12592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:15.686008 12592 log.cc:824] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:15.686537 12592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24: No bootstrap required, opened a new log
I20260926 14:56:15.687947 12592 raft_consensus.cc:348] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "690dd01d5c8f4afca537afccac522c24" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42695 } }
I20260926 14:56:15.688066 12592 raft_consensus.cc:374] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:15.688084 12592 raft_consensus.cc:729] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 690dd01d5c8f4afca537afccac522c24, State: Initialized, Role: FOLLOWER
I20260926 14:56:15.688198 12592 consensus_queue.cc:260] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "690dd01d5c8f4afca537afccac522c24" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42695 } }
I20260926 14:56:15.688275 12592 raft_consensus.cc:388] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:56:15.688308 12592 raft_consensus.cc:482] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:56:15.688338 12592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:15.688747 12592 raft_consensus.cc:504] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "690dd01d5c8f4afca537afccac522c24" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42695 } }
I20260926 14:56:15.688823 12592 leader_election.cc:302] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 690dd01d5c8f4afca537afccac522c24; no voters:
I20260926 14:56:15.688923 12592 leader_election.cc:288] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:56:15.688985 12595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:15.689110 12595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24 [term 1 LEADER]: Becoming Leader. State: Replica: 690dd01d5c8f4afca537afccac522c24, State: Running, Role: LEADER
I20260926 14:56:15.689143 12592 sys_catalog.cc:564] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:15.689196 12595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "690dd01d5c8f4afca537afccac522c24" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42695 } }
I20260926 14:56:15.689565 12596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "690dd01d5c8f4afca537afccac522c24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "690dd01d5c8f4afca537afccac522c24" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42695 } } }
I20260926 14:56:15.689653 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 12525
I20260926 14:56:15.689653 12597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 690dd01d5c8f4afca537afccac522c24. Latest consensus state: current_term: 1 leader_uuid: "690dd01d5c8f4afca537afccac522c24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "690dd01d5c8f4afca537afccac522c24" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42695 } } }
I20260926 14:56:15.689746 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:56:15.689749 12596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:15.689761 12597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:15.689914 7319 external_mini_cluster.cc:1428] Setting key 44945b50c2181413ae0c5dc1c035b8ff
I20260926 14:56:15.690111 12605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:56:15.690583 12605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:56:15.692206 12605 catalog_manager.cc:1347] Generated new cluster ID: 66b97d00433a4f228420267e2656d1fc
I20260926 14:56:15.692256 12605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:56:15.694702 12613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.690354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46056 (local address 127.7.37.254:42695)
0926 14:56:15.690554 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:15.690562 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:15.690590 (+ 28us) server_negotiation.cc:407] Connection header received
0926 14:56:15.690786 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:15.690793 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:15.690856 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:15.690980 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:15.691150 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:15.692303 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:15.693018 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:15.693421 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:15.693518 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:15.694098 (+ 580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:15.694102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:15.694102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:15.694249 (+ 147us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:15.694278 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:15.694287 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:15.694339 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:15.694415 (+ 76us) server_negotiation.cc:299] Negotiation successful
0926 14:56:15.694565 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
I20260926 14:56:15.695412 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:56:15.755874 12605 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:56:15.756352 12605 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:15.759457 12605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 690dd01d5c8f4afca537afccac522c24: Generated new TSK 0
I20260926 14:56:15.759613 12605 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 14:56:15.783253 12622 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260926 14:56:15.783308 12622 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 14:56:15.783630 12613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.769186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46060 (local address 127.7.37.254:42695)
0926 14:56:15.769228 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:15.769230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:15.769289 (+ 59us) server_negotiation.cc:407] Connection header received
0926 14:56:15.769323 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:15.769324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:15.769336 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:15.769423 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0926 14:56:15.769546 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:15.770261 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:15.770998 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:15.771303 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:15.771383 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:15.771542 (+ 159us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 14:56:15.783439 (+ 11897us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 14:56:15.783568 (+ 129us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46060: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":25}
W20260926 14:56:15.783681 12613 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46060: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260926 14:56:15.783679 12621 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:15.769103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42695 (local address 127.0.0.1:46060)
0926 14:56:15.769225 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:15.769234 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:15.769286 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:15.769418 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:15.769419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:15.769420 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0926 14:56:15.769487 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:15.769488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:15.770348 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:15.770349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:15.770918 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:15.770919 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:15.771389 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:15.771390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:15.771432 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:15.771437 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 14:56:15.783535 (+ 12098us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 14:56:15.783605 (+ 70us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.37.254:42695: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
I20260926 14:56:15.783829 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 12525
2026-09-26T14:56:15Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (260 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260926 14:56:15.820042 7319 mini_oidc.cc:152] Starting JWKS server
I20260926 14:56:15.820642 7319 webserver.cc:460] Webserver started at https://127.0.0.1:42669/ using document root <none> and password file <none>
I20260926 14:56:15.820683 7319 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 14:56:15.820807 7319 webserver.cc:460] Webserver started at http://127.0.0.1:37865/ using document root <none> and password file <none>
2026-09-26T14:56:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:15Z Disabled control of system clock
I20260926 14:56:15.826906 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36631
--webserver_interface=127.7.37.254
--webserver_port=0
--builtin_ntp_servers=127.7.37.212:33599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:36631
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42669/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 14:56:15.900697 12628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:15.900866 12628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:15.900892 12628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:15.901971 12628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:15.902007 12628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:15.902024 12628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:15.902040 12628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:15.902055 12628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 14:56:15.902073 12628 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 14:56:15.902092 12628 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42669/jwks.json
I20260926 14:56:15.903378 12628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:33599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:36631
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42669/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:15.903590 12628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:15.903843 12628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 14:56:15.908041 12628 server_base.cc:1034] running on GCE node
W20260926 14:56:15.907990 12634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:15.907999 12633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:15.907994 12636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:15.908396 12628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:15.908636 12628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:15.909782 12628 hybrid_clock.cc:648] HybridClock initialized: now 1790434575909764 us; error 33 us; skew 500 ppm
I20260926 14:56:15.910846 12628 webserver.cc:460] Webserver started at http://127.7.37.254:42757/ using document root <none> and password file <none>
I20260926 14:56:15.911072 12628 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:15.911119 12628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:15.911213 12628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:15.912199 12628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "d8a0b6b1612844cf9252269fb25b6a19"
format_stamp: "Formatted at 2026-09-26 14:56:15 on dist-test-slave-f9xb"
server_key: "6e93092ec75c021ede8ff5f8b016288f"
server_key_iv: "ae163cdc9199a4a23bd19f411bbca50d"
server_key_version: "encryptionkey@0"
I20260926 14:56:15.912509 12628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "d8a0b6b1612844cf9252269fb25b6a19"
format_stamp: "Formatted at 2026-09-26 14:56:15 on dist-test-slave-f9xb"
server_key: "6e93092ec75c021ede8ff5f8b016288f"
server_key_iv: "ae163cdc9199a4a23bd19f411bbca50d"
server_key_version: "encryptionkey@0"
I20260926 14:56:15.913597 12628 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:15.914188 12642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:15.914323 12628 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:15.914419 12628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "d8a0b6b1612844cf9252269fb25b6a19"
format_stamp: "Formatted at 2026-09-26 14:56:15 on dist-test-slave-f9xb"
server_key: "6e93092ec75c021ede8ff5f8b016288f"
server_key_iv: "ae163cdc9199a4a23bd19f411bbca50d"
server_key_version: "encryptionkey@0"
I20260926 14:56:15.914482 12628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:15.932535 12628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:15.932751 12628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:15.932857 12628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:15.936255 12628 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:36631
I20260926 14:56:15.936282 12694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:36631 every 8 connection(s)
I20260926 14:56:15.936546 12628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:56:15.937086 12695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:15.939411 12695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19: Bootstrap starting.
I20260926 14:56:15.939917 12695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:15.940174 12695 log.cc:824] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:15.940207 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 12628
I20260926 14:56:15.940289 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:56:15.940487 7319 external_mini_cluster.cc:1428] Setting key 44b92304ed762834f4a5dfd29a3c02a5
I20260926 14:56:15.940801 12695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19: No bootstrap required, opened a new log
I20260926 14:56:15.942396 12695 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8a0b6b1612844cf9252269fb25b6a19" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36631 } }
I20260926 14:56:15.942514 12695 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:15.942544 12695 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8a0b6b1612844cf9252269fb25b6a19, State: Initialized, Role: FOLLOWER
I20260926 14:56:15.942651 12695 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8a0b6b1612844cf9252269fb25b6a19" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36631 } }
I20260926 14:56:15.942718 12695 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:56:15.942746 12695 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:56:15.942792 12695 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:15.943248 12695 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8a0b6b1612844cf9252269fb25b6a19" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36631 } }
I20260926 14:56:15.943323 12695 leader_election.cc:302] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8a0b6b1612844cf9252269fb25b6a19; no voters:
I20260926 14:56:15.943483 12695 leader_election.cc:288] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:56:15.943531 12700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:15.943629 12700 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19 [term 1 LEADER]: Becoming Leader. State: Replica: d8a0b6b1612844cf9252269fb25b6a19, State: Running, Role: LEADER
I20260926 14:56:15.943711 12695 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:15.943701 12700 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8a0b6b1612844cf9252269fb25b6a19" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36631 } }
I20260926 14:56:15.944027 12702 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8a0b6b1612844cf9252269fb25b6a19. Latest consensus state: current_term: 1 leader_uuid: "d8a0b6b1612844cf9252269fb25b6a19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8a0b6b1612844cf9252269fb25b6a19" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36631 } } }
I20260926 14:56:15.944103 12702 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:15.944233 12701 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8a0b6b1612844cf9252269fb25b6a19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8a0b6b1612844cf9252269fb25b6a19" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36631 } } }
I20260926 14:56:15.944291 12701 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19 [sys.catalog]: This master's current role is: LEADER
W20260926 14:56:15.945247 12716 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 14:56:15.945302 12716 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 14:56:15.945984 12706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:56:15.946132 12706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:56:15.947029 12699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.940902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43036 (local address 127.7.37.254:36631)
0926 14:56:15.941100 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:15.941109 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:15.941131 (+ 22us) server_negotiation.cc:407] Connection header received
0926 14:56:15.942375 (+ 1244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:15.942381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:15.942458 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:15.942575 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:15.942765 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:15.944181 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:15.945242 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:15.945670 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:15.945773 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:15.946372 (+ 599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:15.946376 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:15.946376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:15.946555 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:15.946594 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:15.946608 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:15.946682 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:15.946747 (+ 65us) server_negotiation.cc:299] Negotiation successful
0926 14:56:15.946822 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":68,"threads_started":1}
I20260926 14:56:15.947520 12706 catalog_manager.cc:1347] Generated new cluster ID: 5882a6685bbc4e6bba4e0d0a79630935
I20260926 14:56:15.947568 12706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:56:15.947841 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:56:15.954640 12706 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:56:15.955067 12706 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:15.961376 12706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8a0b6b1612844cf9252269fb25b6a19: Generated new TSK 0
I20260926 14:56:15.961505 12706 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:56:15.968168 12699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:15.965326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43048 (local address 127.7.37.254:36631)
0926 14:56:15.965365 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:15.965367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:15.965455 (+ 88us) server_negotiation.cc:407] Connection header received
0926 14:56:15.965521 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:15.965522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:15.965540 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:15.965596 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0926 14:56:15.965702 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:15.966152 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:15.966861 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:15.967048 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:15.967099 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:15.967233 (+ 134us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 14:56:15.967998 (+ 765us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42669/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 14:56:15.968111 (+ 113us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43048: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42669/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":22}
W20260926 14:56:15.968206 12699 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43048: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42669/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260926 14:56:15.968276 12724 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:15.965277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36631 (local address 127.0.0.1:43048)
0926 14:56:15.965397 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:15.965406 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:15.965461 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:15.965599 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:15.965600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:15.965602 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0926 14:56:15.965646 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:15.965648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:15.966227 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:15.966227 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:15.966792 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:15.966794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:15.967105 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:15.967106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:15.967158 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:15.967162 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 14:56:15.968091 (+ 929us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42669/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 14:56:15.968184 (+ 93us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.37.254:36631: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42669/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":80,"threads_started":1}
I20260926 14:56:15.968403 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 12628
E20260926 14:56:15.980207 12725 webserver.cc:533] Webserver: sslize failed:
E20260926 14:56:15.980257 12725 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-26T14:56:15Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (185 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-26T14:56:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:16Z Disabled control of system clock
I20260926 14:56:16.012526 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:38413
--webserver_interface=127.7.37.254
--webserver_port=0
--builtin_ntp_servers=127.7.37.212:45547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:38413
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434555014799-7319-0/testchainca.cert with env {}
W20260926 14:56:16.086103 12729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:16.086272 12729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:16.086292 12729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:16.087401 12729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:16.087438 12729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:16.087450 12729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:16.087460 12729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:16.087474 12729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 14:56:16.087498 12729 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 14:56:16.087522 12729 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260926 14:56:16.088963 12729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:45547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:38413
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:38413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434555014799-7319-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:16.089164 12729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:16.089344 12729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:16.093513 12735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:16.093528 12734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:16.093528 12737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:16.093672 12729 server_base.cc:1034] running on GCE node
I20260926 14:56:16.093919 12729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:16.094163 12729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:16.095319 12729 hybrid_clock.cc:648] HybridClock initialized: now 1790434576095307 us; error 26 us; skew 500 ppm
I20260926 14:56:16.096441 12729 webserver.cc:460] Webserver started at http://127.7.37.254:42825/ using document root <none> and password file <none>
I20260926 14:56:16.096633 12729 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:16.096676 12729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:16.096781 12729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:16.097675 12729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "c45090718ffa44e1a40b5cb18f192f81"
format_stamp: "Formatted at 2026-09-26 14:56:16 on dist-test-slave-f9xb"
server_key: "551baca2bc939aa435e4f9da8856f3cd"
server_key_iv: "4e46f9958902fde1cd8efb70a1dd93a8"
server_key_version: "encryptionkey@0"
I20260926 14:56:16.097963 12729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "c45090718ffa44e1a40b5cb18f192f81"
format_stamp: "Formatted at 2026-09-26 14:56:16 on dist-test-slave-f9xb"
server_key: "551baca2bc939aa435e4f9da8856f3cd"
server_key_iv: "4e46f9958902fde1cd8efb70a1dd93a8"
server_key_version: "encryptionkey@0"
I20260926 14:56:16.098999 12729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:16.099581 12743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:16.099700 12729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:16.099789 12729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "c45090718ffa44e1a40b5cb18f192f81"
format_stamp: "Formatted at 2026-09-26 14:56:16 on dist-test-slave-f9xb"
server_key: "551baca2bc939aa435e4f9da8856f3cd"
server_key_iv: "4e46f9958902fde1cd8efb70a1dd93a8"
server_key_version: "encryptionkey@0"
I20260926 14:56:16.099843 12729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:16.128950 12729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:16.129194 12729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:16.129309 12729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:16.132709 12729 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:38413
I20260926 14:56:16.132738 12795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:38413 every 8 connection(s)
I20260926 14:56:16.133013 12729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:56:16.133831 12796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:16.135943 12796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81: Bootstrap starting.
I20260926 14:56:16.135988 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 12729
I20260926 14:56:16.136075 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:56:16.136242 7319 external_mini_cluster.cc:1428] Setting key 7f31868896b9b08e1fced3f0a27cd9e7
I20260926 14:56:16.136574 12796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:16.136875 12796 log.cc:824] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:16.137542 12796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81: No bootstrap required, opened a new log
I20260926 14:56:16.139457 12796 raft_consensus.cc:348] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45090718ffa44e1a40b5cb18f192f81" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 38413 } }
I20260926 14:56:16.139611 12796 raft_consensus.cc:374] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:16.139644 12796 raft_consensus.cc:729] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c45090718ffa44e1a40b5cb18f192f81, State: Initialized, Role: FOLLOWER
I20260926 14:56:16.139791 12796 consensus_queue.cc:260] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45090718ffa44e1a40b5cb18f192f81" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 38413 } }
I20260926 14:56:16.139868 12796 raft_consensus.cc:388] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:56:16.139897 12796 raft_consensus.cc:482] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:56:16.139940 12796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:16.140473 12796 raft_consensus.cc:504] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45090718ffa44e1a40b5cb18f192f81" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 38413 } }
I20260926 14:56:16.140570 12796 leader_election.cc:302] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c45090718ffa44e1a40b5cb18f192f81; no voters:
I20260926 14:56:16.140707 12796 leader_election.cc:288] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:56:16.140736 12801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:16.140873 12801 raft_consensus.cc:686] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81 [term 1 LEADER]: Becoming Leader. State: Replica: c45090718ffa44e1a40b5cb18f192f81, State: Running, Role: LEADER
I20260926 14:56:16.140926 12796 sys_catalog.cc:564] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:16.140955 12801 consensus_queue.cc:237] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45090718ffa44e1a40b5cb18f192f81" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 38413 } }
I20260926 14:56:16.141265 12802 sys_catalog.cc:455] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c45090718ffa44e1a40b5cb18f192f81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45090718ffa44e1a40b5cb18f192f81" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 38413 } } }
I20260926 14:56:16.141260 12803 sys_catalog.cc:455] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c45090718ffa44e1a40b5cb18f192f81. Latest consensus state: current_term: 1 leader_uuid: "c45090718ffa44e1a40b5cb18f192f81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45090718ffa44e1a40b5cb18f192f81" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 38413 } } }
I20260926 14:56:16.141345 12802 sys_catalog.cc:458] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:16.141350 12803 sys_catalog.cc:458] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:16.141600 12808 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:56:16.141916 12799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:16.136672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48742 (local address 127.7.37.254:38413)
0926 14:56:16.136846 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:16.136853 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:16.136875 (+ 22us) server_negotiation.cc:407] Connection header received
0926 14:56:16.137388 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:16.137396 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:16.137478 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:16.137593 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:16.137796 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:16.138646 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:16.139703 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:16.140200 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:16.140290 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:16.141133 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:16.141137 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:16.141137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:16.141321 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:16.141363 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:16.141377 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:16.141417 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:16.141646 (+ 229us) server_negotiation.cc:299] Negotiation successful
0926 14:56:16.141725 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260926 14:56:16.142164 12808 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:56:16.143260 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:56:16.143564 12808 catalog_manager.cc:1347] Generated new cluster ID: 5b121678184041e5b6a7f6dcdc82698b
I20260926 14:56:16.143602 12808 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:56:16.156078 12808 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:56:16.156515 12808 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:16.159934 12808 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c45090718ffa44e1a40b5cb18f192f81: Generated new TSK 0
I20260926 14:56:16.160080 12808 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:56:16.209856 12799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:16.166324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48758 (local address 127.7.37.254:38413)
0926 14:56:16.166361 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:16.166362 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:16.166452 (+ 90us) server_negotiation.cc:407] Connection header received
0926 14:56:16.166512 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:16.166513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:16.166521 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:16.166571 (+ 50us) server_negotiation.cc:226] Negotiated authn=JWT
0926 14:56:16.166683 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:16.167134 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:16.168189 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:16.168370 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:16.168417 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:16.168560 (+ 143us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 14:56:16.209692 (+ 41132us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 14:56:16.209807 (+ 115us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48758: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":21}
W20260926 14:56:16.209908 12799 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48758: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260926 14:56:16.210052 12825 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:16.166259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:38413 (local address 127.0.0.1:48758)
0926 14:56:16.166383 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:16.166392 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:16.166453 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:16.166580 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:16.166581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:16.166583 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0926 14:56:16.166632 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:16.166634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:16.167191 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:16.167192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:16.168122 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:16.168125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:16.168431 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:16.168431 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:16.168487 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:16.168492 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 14:56:16.209816 (+ 41324us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 14:56:16.209945 (+ 129us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.37.254:38413: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":70,"threads_started":1}
I20260926 14:56:16.210301 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 12729
2026-09-26T14:56:16Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (220 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (80 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (80 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:56:16 dist-test-slave-f9xb krb5kdc[12837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:56:16 dist-test-slave-f9xb krb5kdc[12837](info): set up 2 sockets
Sep 26 14:56:16 dist-test-slave-f9xb krb5kdc[12837](info): commencing operation
krb5kdc: starting...
W20260926 14:56:17.307410 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.920s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:56:17 dist-test-slave-f9xb krb5kdc[12837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:56:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:56:17.412087 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:39889
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:39889
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:17.486615 12850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:17.486789 12850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:17.486819 12850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:17.487952 12850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:17.487990 12850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:17.488009 12850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:17.488025 12850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:17.488040 12850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:56:17.489451 12850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:39889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:39889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:17.489660 12850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:17.489832 12850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:17.493999 12858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:17.494071 12855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:17.494156 12850 server_base.cc:1034] running on GCE node
W20260926 14:56:17.494082 12856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:17.494524 12850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:17.494762 12850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:17.495896 12850 hybrid_clock.cc:648] HybridClock initialized: now 1790434577495884 us; error 30 us; skew 500 ppm
Sep 26 14:56:17 dist-test-slave-f9xb krb5kdc[12837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434577, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:17.497932 12850 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:56:17.498270 12850 webserver.cc:460] Webserver started at http://127.7.37.254:34163/ using document root <none> and password file <none>
I20260926 14:56:17.498416 12850 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:17.498451 12850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:17.498528 12850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:17.499374 12850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "c61c1d0972f346d5aba0cc124e626135"
format_stamp: "Formatted at 2026-09-26 14:56:17 on dist-test-slave-f9xb"
server_key: "69dcf68765410c647690473087252d56"
server_key_iv: "017fe2b1e99521e13ff2c57163169cc0"
server_key_version: "encryptionkey@0"
I20260926 14:56:17.499629 12850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "c61c1d0972f346d5aba0cc124e626135"
format_stamp: "Formatted at 2026-09-26 14:56:17 on dist-test-slave-f9xb"
server_key: "69dcf68765410c647690473087252d56"
server_key_iv: "017fe2b1e99521e13ff2c57163169cc0"
server_key_version: "encryptionkey@0"
I20260926 14:56:17.500702 12850 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:17.501260 12865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:17.501392 12850 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:17.501463 12850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "c61c1d0972f346d5aba0cc124e626135"
format_stamp: "Formatted at 2026-09-26 14:56:17 on dist-test-slave-f9xb"
server_key: "69dcf68765410c647690473087252d56"
server_key_iv: "017fe2b1e99521e13ff2c57163169cc0"
server_key_version: "encryptionkey@0"
I20260926 14:56:17.501513 12850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:17.519605 12850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:17.521121 12850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:17.521211 12850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:17.524431 12850 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:39889
I20260926 14:56:17.524461 12917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:39889 every 8 connection(s)
I20260926 14:56:17.524725 12850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:56:17.525164 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 12850
I20260926 14:56:17.525250 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:56:17.525418 7319 external_mini_cluster.cc:1428] Setting key 43f6dcad4f6b264e5cba6d1aad0f077c
I20260926 14:56:17.525477 12918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:17.527812 12918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135: Bootstrap starting.
I20260926 14:56:17.528308 12918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:17.528544 12918 log.cc:824] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:17.529078 12918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135: No bootstrap required, opened a new log
Sep 26 14:56:17 dist-test-slave-f9xb krb5kdc[12837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:17.530772 12918 raft_consensus.cc:348] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61c1d0972f346d5aba0cc124e626135" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39889 } }
I20260926 14:56:17.530915 12918 raft_consensus.cc:374] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:17.530956 12918 raft_consensus.cc:729] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c61c1d0972f346d5aba0cc124e626135, State: Initialized, Role: FOLLOWER
I20260926 14:56:17.531075 12918 consensus_queue.cc:260] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61c1d0972f346d5aba0cc124e626135" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39889 } }
I20260926 14:56:17.531143 12918 raft_consensus.cc:388] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:56:17.531178 12918 raft_consensus.cc:482] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:56:17.531214 12918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:17.531790 12918 raft_consensus.cc:504] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61c1d0972f346d5aba0cc124e626135" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39889 } }
I20260926 14:56:17.531884 12918 leader_election.cc:302] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c61c1d0972f346d5aba0cc124e626135; no voters:
I20260926 14:56:17.532027 12918 leader_election.cc:288] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:56:17.532070 12923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:17.532179 12923 raft_consensus.cc:686] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135 [term 1 LEADER]: Becoming Leader. State: Replica: c61c1d0972f346d5aba0cc124e626135, State: Running, Role: LEADER
I20260926 14:56:17.532241 12918 sys_catalog.cc:564] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:17.532264 12923 consensus_queue.cc:237] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61c1d0972f346d5aba0cc124e626135" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39889 } }
I20260926 14:56:17.532634 12924 sys_catalog.cc:455] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c61c1d0972f346d5aba0cc124e626135" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61c1d0972f346d5aba0cc124e626135" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39889 } } }
I20260926 14:56:17.532645 12925 sys_catalog.cc:455] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c61c1d0972f346d5aba0cc124e626135. Latest consensus state: current_term: 1 leader_uuid: "c61c1d0972f346d5aba0cc124e626135" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61c1d0972f346d5aba0cc124e626135" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39889 } } }
I20260926 14:56:17.532716 12924 sys_catalog.cc:458] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:17.532738 12925 sys_catalog.cc:458] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:17.533464 12931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:56:17.533913 12931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:56:17.534551 12921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:17.525851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37438 (local address 127.7.37.254:39889)
0926 14:56:17.526015 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:17.526021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:17.526043 (+ 22us) server_negotiation.cc:407] Connection header received
0926 14:56:17.526378 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:17.526383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:17.526430 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:17.526542 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:17.527086 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:17.528010 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:17.528722 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:17.529110 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:17.529199 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:17.531216 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:17.531220 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:17.531220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:17.531230 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:17.532929 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:17.533238 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:17.533241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:17.533241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:17.533334 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:17.533510 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:17.533511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:17.533511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:17.533783 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:17.533906 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:17.534249 (+ 343us) server_negotiation.cc:299] Negotiation successful
0926 14:56:17.534367 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
I20260926 14:56:17.535373 12931 catalog_manager.cc:1347] Generated new cluster ID: 2d60c13d37ea46f2914f0c2f885d0e10
I20260926 14:56:17.535419 12931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:56:17.560328 12931 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:56:17.560972 12931 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:17.565558 12931 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c61c1d0972f346d5aba0cc124e626135: Generated new TSK 0
I20260926 14:56:17.565694 12931 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
I20260926 14:56:17.592133 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:39889
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:17.666951 12944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:17.667126 12944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:17.667155 12944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:17.668282 12944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:17.668319 12944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:17.668380 12944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:56:17.669759 12944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:39889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:17.669960 12944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:17.670137 12944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:17.674503 12950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:17.674543 12944 server_base.cc:1034] running on GCE node
W20260926 14:56:17.674503 12952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:17.674515 12949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:17.674852 12944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:17.675073 12944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:17.676198 12944 hybrid_clock.cc:648] HybridClock initialized: now 1790434577676188 us; error 31 us; skew 500 ppm
Sep 26 14:56:17 dist-test-slave-f9xb krb5kdc[12837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434577, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:17.678391 12944 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:56:17.678759 12944 webserver.cc:460] Webserver started at http://127.7.37.193:38045/ using document root <none> and password file <none>
I20260926 14:56:17.678932 12944 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:17.678977 12944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:17.679086 12944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:17.680011 12944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "c959cceb1d974c6a83b1118c767def69"
format_stamp: "Formatted at 2026-09-26 14:56:17 on dist-test-slave-f9xb"
server_key: "cb33488f217884202e5f8c6a2fcd4c78"
server_key_iv: "1b9e8be63fcd5cf501e76be3f9c606d7"
server_key_version: "encryptionkey@0"
I20260926 14:56:17.680298 12944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "c959cceb1d974c6a83b1118c767def69"
format_stamp: "Formatted at 2026-09-26 14:56:17 on dist-test-slave-f9xb"
server_key: "cb33488f217884202e5f8c6a2fcd4c78"
server_key_iv: "1b9e8be63fcd5cf501e76be3f9c606d7"
server_key_version: "encryptionkey@0"
I20260926 14:56:17.681423 12944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:17.682044 12959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:17.682180 12944 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:17.682252 12944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "c959cceb1d974c6a83b1118c767def69"
format_stamp: "Formatted at 2026-09-26 14:56:17 on dist-test-slave-f9xb"
server_key: "cb33488f217884202e5f8c6a2fcd4c78"
server_key_iv: "1b9e8be63fcd5cf501e76be3f9c606d7"
server_key_version: "encryptionkey@0"
I20260926 14:56:17.682307 12944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:17.699615 12944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:17.701176 12944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:17.701292 12944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:17.701463 12944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:17.701713 12944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:17.701743 12944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:17.701767 12944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:17.701784 12944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:17.706895 12944 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:38403
I20260926 14:56:17.706957 13072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:38403 every 8 connection(s)
I20260926 14:56:17.707181 12944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
Sep 26 14:56:17 dist-test-slave-f9xb krb5kdc[12837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434577, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:17.715095 12921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:17.707793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:56207 (local address 127.7.37.254:39889)
0926 14:56:17.707830 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:17.707832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:17.708004 (+ 172us) server_negotiation.cc:407] Connection header received
0926 14:56:17.708825 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:17.708826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:17.708838 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:17.708888 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:17.709408 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:17.709891 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:17.710814 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:17.711029 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:17.711096 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:17.712922 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:17.712923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:17.712924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:17.712930 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:17.713959 (+ 1029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:17.714286 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:17.714287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:17.714287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:17.714339 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:17.714542 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:17.714543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:17.714543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:17.714643 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:17.714694 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:17.714933 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 14:56:17.715011 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 14:56:17.715207 13075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:17.707777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:39889 (local address 127.7.37.193:56207)
0926 14:56:17.707903 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:17.707933 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:17.708729 (+ 796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:17.708944 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:17.708947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:17.709170 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:17.709327 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:17.709331 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:17.709973 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:17.709974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:17.710740 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:17.710742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:17.711107 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:17.711108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:17.711217 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:17.711601 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:17.711608 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:17.712836 (+ 1228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:17.714027 (+ 1191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:17.714032 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:17.714034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:17.714224 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:17.714391 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:17.714392 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:17.714392 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:17.714490 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:17.714694 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:17.714700 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:17.714818 (+ 118us) client_negotiation.cc:769] Sending connection context
0926 14:56:17.714898 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 14:56:17.715026 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":64,"threads_started":1}
I20260926 14:56:17.715662 13073 heartbeater.cc:347] Connected to a master server at 127.7.37.254:39889
I20260926 14:56:17.715736 13073 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:17.715934 13073 heartbeater.cc:511] Master 127.7.37.254:39889 requested a full tablet report, sending...
I20260926 14:56:17.715934 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 12944
I20260926 14:56:17.716020 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:56:17.716194 7319 external_mini_cluster.cc:1428] Setting key e11962a50b52ae0a0475a64005e76652
I20260926 14:56:17.716434 12882 ts_manager.cc:194] Registered new tserver with Master: c959cceb1d974c6a83b1118c767def69 (127.7.37.193:38403)
I20260926 14:56:17.717175 12882 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:56207
WARNING: no policy specified for kudu/127.7.37.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
I20260926 14:56:17.736569 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:39889
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:17.811002 13078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:17.811183 13078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:17.811203 13078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:17.812314 13078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:17.812350 13078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:17.812374 13078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:56:17.813735 13078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:39889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:17.813930 13078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:17.814116 13078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:17.818415 13086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:17.818427 13084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:17.818490 13083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:17.818583 13078 server_base.cc:1034] running on GCE node
I20260926 14:56:17.818810 13078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:17.819001 13078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:17.820117 13078 hybrid_clock.cc:648] HybridClock initialized: now 1790434577820109 us; error 25 us; skew 500 ppm
Sep 26 14:56:17 dist-test-slave-f9xb krb5kdc[12837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434577, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:17.822252 13078 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:56:17.822585 13078 webserver.cc:460] Webserver started at http://127.7.37.194:33327/ using document root <none> and password file <none>
I20260926 14:56:17.822746 13078 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:17.822782 13078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:17.822863 13078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:17.823719 13078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "5db132b9e7c9422e91b060ff409e02e7"
format_stamp: "Formatted at 2026-09-26 14:56:17 on dist-test-slave-f9xb"
server_key: "32b8795d283219487aff49a813808adf"
server_key_iv: "1bf68dd5a75c85ffa789ffebc0b59805"
server_key_version: "encryptionkey@0"
I20260926 14:56:17.824023 13078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "5db132b9e7c9422e91b060ff409e02e7"
format_stamp: "Formatted at 2026-09-26 14:56:17 on dist-test-slave-f9xb"
server_key: "32b8795d283219487aff49a813808adf"
server_key_iv: "1bf68dd5a75c85ffa789ffebc0b59805"
server_key_version: "encryptionkey@0"
I20260926 14:56:17.825071 13078 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:17.825662 13093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:17.825793 13078 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:17.825865 13078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "5db132b9e7c9422e91b060ff409e02e7"
format_stamp: "Formatted at 2026-09-26 14:56:17 on dist-test-slave-f9xb"
server_key: "32b8795d283219487aff49a813808adf"
server_key_iv: "1bf68dd5a75c85ffa789ffebc0b59805"
server_key_version: "encryptionkey@0"
I20260926 14:56:17.825912 13078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:17.851907 13078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:17.853385 13078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:17.853477 13078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:17.853616 13078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:17.853817 13078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:17.853844 13078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:17.853861 13078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:17.853878 13078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:17.858785 13078 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:38455
I20260926 14:56:17.858847 13206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:38455 every 8 connection(s)
I20260926 14:56:17.859068 13078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:56:17.859997 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 13078
I20260926 14:56:17.860087 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:56:17.860245 7319 external_mini_cluster.cc:1428] Setting key 189253770218336250d5638239aaa0f5
Sep 26 14:56:17 dist-test-slave-f9xb krb5kdc[12837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434577, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.37.195@KRBTEST.COM; defaulting to no policy
I20260926 14:56:17.868052 12921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:17.859689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:37093 (local address 127.7.37.254:39889)
0926 14:56:17.859758 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:17.859760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:17.859919 (+ 159us) server_negotiation.cc:407] Connection header received
0926 14:56:17.860783 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:17.860784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:17.860794 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:17.860846 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:17.861345 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:17.862022 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:17.863321 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:17.863500 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:17.863541 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:17.865659 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:17.865660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:17.865661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:17.865667 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:17.866838 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:17.867255 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:17.867255 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:17.867256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:17.867297 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:17.867504 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:17.867504 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:17.867505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:17.867589 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:17.867629 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:17.867939 (+ 310us) server_negotiation.cc:299] Negotiation successful
0926 14:56:17.867994 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 14:56:17.868228 13209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:17.859656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:39889 (local address 127.7.37.194:37093)
0926 14:56:17.859814 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:17.859847 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:17.860691 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:17.860892 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:17.860894 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:17.861104 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:17.861263 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:17.861267 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:17.862107 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:17.862108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:17.863249 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:17.863251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:17.863560 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:17.863561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:17.863690 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:17.864142 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:17.864151 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:17.865578 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:17.866913 (+ 1335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:17.866928 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:17.866930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:17.867188 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:17.867335 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:17.867335 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:17.867336 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:17.867454 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:17.867642 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:17.867647 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:17.867811 (+ 164us) client_negotiation.cc:769] Sending connection context
0926 14:56:17.867930 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 14:56:17.868054 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":95,"threads_started":1}
I20260926 14:56:17.868676 13207 heartbeater.cc:347] Connected to a master server at 127.7.37.254:39889
I20260926 14:56:17.868752 13207 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:17.868893 13207 heartbeater.cc:511] Master 127.7.37.254:39889 requested a full tablet report, sending...
I20260926 14:56:17.869163 12882 ts_manager.cc:194] Registered new tserver with Master: 5db132b9e7c9422e91b060ff409e02e7 (127.7.37.194:38455)
I20260926 14:56:17.869544 12882 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:37093
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
I20260926 14:56:17.880558 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:39889
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:17.953743 13212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:17.953908 13212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:17.953938 13212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:17.955021 13212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:17.955060 13212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:17.955093 13212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:56:17.956453 13212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:39889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:17.956645 13212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:17.956823 13212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:17.961076 13218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:17.961078 13217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:17.961076 13220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:17.961292 13212 server_base.cc:1034] running on GCE node
I20260926 14:56:17.961467 13212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:17.961628 13212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:17.962752 13212 hybrid_clock.cc:648] HybridClock initialized: now 1790434577962740 us; error 26 us; skew 500 ppm
Sep 26 14:56:17 dist-test-slave-f9xb krb5kdc[12837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434577, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:17.964849 13212 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:56:17.965193 13212 webserver.cc:460] Webserver started at http://127.7.37.195:44449/ using document root <none> and password file <none>
I20260926 14:56:17.965354 13212 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:17.965392 13212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:17.965500 13212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:17.966346 13212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "511e27b2e8cf47b490082358369d9ecb"
format_stamp: "Formatted at 2026-09-26 14:56:17 on dist-test-slave-f9xb"
server_key: "65ec7921177fa0d3d83728b9ac9834c6"
server_key_iv: "f832f259337d21d5a9b1a609b5fe1cc6"
server_key_version: "encryptionkey@0"
I20260926 14:56:17.966642 13212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "511e27b2e8cf47b490082358369d9ecb"
format_stamp: "Formatted at 2026-09-26 14:56:17 on dist-test-slave-f9xb"
server_key: "65ec7921177fa0d3d83728b9ac9834c6"
server_key_iv: "f832f259337d21d5a9b1a609b5fe1cc6"
server_key_version: "encryptionkey@0"
I20260926 14:56:17.967703 13212 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 14:56:17.968286 13227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:17.968417 13212 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:17.968497 13212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "511e27b2e8cf47b490082358369d9ecb"
format_stamp: "Formatted at 2026-09-26 14:56:17 on dist-test-slave-f9xb"
server_key: "65ec7921177fa0d3d83728b9ac9834c6"
server_key_iv: "f832f259337d21d5a9b1a609b5fe1cc6"
server_key_version: "encryptionkey@0"
I20260926 14:56:17.968554 13212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:17.975864 13212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:17.977267 13212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:17.977368 13212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:17.977540 13212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:17.977761 13212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:17.977790 13212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:17.977807 13212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:17.977828 13212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:17.982765 13212 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:34019
I20260926 14:56:17.982820 13340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:34019 every 8 connection(s)
I20260926 14:56:17.983022 13212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:56:17.983640 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 13212
I20260926 14:56:17.983732 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:56:17.983927 7319 external_mini_cluster.cc:1428] Setting key 4fc6530b3d558af9f21d029386b21eec
Sep 26 14:56:17 dist-test-slave-f9xb krb5kdc[12837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434577, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:17.990835 12921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:17.983584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:59249 (local address 127.7.37.254:39889)
0926 14:56:17.983633 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:17.983635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:17.983819 (+ 184us) server_negotiation.cc:407] Connection header received
0926 14:56:17.984742 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:17.984743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:17.984752 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:17.984785 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:17.985281 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:17.985744 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:17.986659 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:17.986880 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:17.986929 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:17.988611 (+ 1682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:17.988612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:17.988612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:17.988618 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:17.989804 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:17.990155 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:17.990157 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:17.990157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:17.990214 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:17.990415 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:17.990415 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:17.990416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:17.990506 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:17.990549 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:17.990718 (+ 169us) server_negotiation.cc:299] Negotiation successful
0926 14:56:17.990772 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 14:56:17.990965 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:17.983544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:39889 (local address 127.7.37.195:59249)
0926 14:56:17.983682 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:17.983725 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:17.984646 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:17.984843 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:17.984846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:17.985061 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:17.985218 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:17.985221 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:17.985798 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:17.985799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:17.986595 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:17.986597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:17.986935 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:17.986935 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:17.987016 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:17.987374 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:17.987381 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:17.988539 (+ 1158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:17.989869 (+ 1330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:17.989874 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:17.989875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:17.990070 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:17.990275 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:17.990275 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:17.990275 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:17.990368 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:17.990546 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:17.990548 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:17.990638 (+ 90us) client_negotiation.cc:769] Sending connection context
0926 14:56:17.990704 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 14:56:17.990800 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":72,"threads_started":1}
I20260926 14:56:17.991343 13341 heartbeater.cc:347] Connected to a master server at 127.7.37.254:39889
I20260926 14:56:17.991413 13341 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:17.991524 13341 heartbeater.cc:511] Master 127.7.37.254:39889 requested a full tablet report, sending...
I20260926 14:56:17.991832 12882 ts_manager.cc:194] Registered new tserver with Master: 511e27b2e8cf47b490082358369d9ecb (127.7.37.195:34019)
I20260926 14:56:17.992168 12882 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:59249
I20260926 14:56:17.995322 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/test-admin.keytab.
Sep 26 14:56:18 dist-test-slave-f9xb krb5kdc[12837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:18.007301 7319 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260926 14:56:18.008617 13348 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:18.007997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40316 (local address 127.7.37.193:38403)
0926 14:56:18.008144 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:18.008147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:18.008156 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:56:18.008176 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:18.008178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:18.008189 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:18.008247 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:18.008549 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40316: BlockingRecv error: recv got EOF from 127.0.0.1:40316 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
W20260926 14:56:18.008744 7319 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/krb5cc' of type FILE
Sep 26 14:56:18 dist-test-slave-f9xb krb5kdc[12837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:18.009582 7319 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 14:56:18 dist-test-slave-f9xb krb5kdc[12837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:18.010346 7319 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 14:56:18 dist-test-slave-f9xb krb5kdc[12837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.193@KRBTEST.COM
I20260926 14:56:18.017045 13348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:18.010757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40330 (local address 127.7.37.193:38403)
0926 14:56:18.010803 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:18.010805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:18.010852 (+ 47us) server_negotiation.cc:407] Connection header received
0926 14:56:18.010896 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:18.010897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:18.010905 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:18.010949 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:18.011343 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:18.012061 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:18.012735 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:18.013033 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:18.013090 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:18.014923 (+ 1833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:18.014929 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:18.014929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:18.014935 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:18.016216 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:18.016481 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:18.016482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:18.016483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:18.016527 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:18.016648 (+ 121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:18.016648 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:18.016649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:18.016746 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:18.016833 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:18.016893 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 14:56:18.016954 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260926 14:56:18.018476 13348 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:18.017998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40336 (local address 127.7.37.193:38403)
0926 14:56:18.018031 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:18.018032 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:18.018077 (+ 45us) server_negotiation.cc:407] Connection header received
0926 14:56:18.018122 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:18.018123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:18.018131 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:18.018169 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:18.018443 (+ 274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40336: BlockingRecv error: recv got EOF from 127.0.0.1:40336 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 14:56:18.018584 7319 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/krb5cc' of type FILE
Sep 26 14:56:18 dist-test-slave-f9xb krb5kdc[12837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:18.019393 7319 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 14:56:18 dist-test-slave-f9xb krb5kdc[12837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:18.020154 7319 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 14:56:18 dist-test-slave-f9xb krb5kdc[12837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.193@KRBTEST.COM
I20260926 14:56:18.025892 13348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:18.020539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40350 (local address 127.7.37.193:38403)
0926 14:56:18.020575 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:18.020576 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:18.020631 (+ 55us) server_negotiation.cc:407] Connection header received
0926 14:56:18.020693 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:18.020693 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:18.020702 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:18.020746 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:18.021052 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:18.021502 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:18.022185 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:18.022378 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:18.022427 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:18.024084 (+ 1657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:18.024085 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:18.024086 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:18.024092 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:18.025040 (+ 948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:18.025298 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:18.025299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:18.025299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:18.025341 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:18.025490 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:18.025491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:18.025491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:18.025570 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:18.025642 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:18.025738 (+ 96us) server_negotiation.cc:299] Negotiation successful
0926 14:56:18.025803 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 14:56:18.027064 13348 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:18.026642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40354 (local address 127.7.37.193:38403)
0926 14:56:18.026693 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:18.026695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:18.026702 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:18.026751 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:18.026752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:18.026759 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:18.026807 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:18.027028 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40354: BlockingRecv error: recv got EOF from 127.0.0.1:40354 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260926 14:56:18.027128 7319 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434555014799-7319-0/krb5kdc/krb5cc' of type FILE
Sep 26 14:56:18 dist-test-slave-f9xb krb5kdc[12837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:18.027941 7319 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 14:56:18 dist-test-slave-f9xb krb5kdc[12837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:18.028641 7319 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 14:56:18 dist-test-slave-f9xb krb5kdc[12837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.193@KRBTEST.COM
I20260926 14:56:18.034618 13348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:18.029098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40366 (local address 127.7.37.193:38403)
0926 14:56:18.029131 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:18.029133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:18.029185 (+ 52us) server_negotiation.cc:407] Connection header received
0926 14:56:18.029250 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:18.029251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:18.029258 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:18.029303 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:18.029590 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:18.030000 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:18.030580 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:18.030740 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:18.030779 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:18.032553 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:18.032554 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:18.032554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:18.032560 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:18.033743 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:18.034009 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:18.034010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:18.034010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:18.034070 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:18.034219 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:18.034220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:18.034220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:18.034307 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:18.034377 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:18.034478 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 14:56:18.034541 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 14:56:18.035065 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 12944
I20260926 14:56:18.038568 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 13078
I20260926 14:56:18.042694 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 13212
I20260926 14:56:18.046041 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 12850
2026-09-26T14:56:18Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1676 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:56:18 dist-test-slave-f9xb krb5kdc[13361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:56:18 dist-test-slave-f9xb krb5kdc[13361](info): set up 2 sockets
Sep 26 14:56:18 dist-test-slave-f9xb krb5kdc[13361](info): commencing operation
krb5kdc: starting...
W20260926 14:56:18.991482 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:56:19 dist-test-slave-f9xb krb5kdc[13361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:56:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:19Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.254.keytab.
Entry for principal oryx/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.254.keytab.
I20260926 14:56:19.093554 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:34241
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.254.keytab
--principal=oryx/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:34241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:19.168136 13374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:19.168311 13374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:19.168339 13374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:19.169437 13374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:19.169476 13374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:19.169493 13374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:19.169509 13374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:19.169525 13374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:56:19.170882 13374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:34241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.254.keytab
--principal=oryx/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:34241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:19.171105 13374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:19.171286 13374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:19.175467 13380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:19.175453 13382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:19.175457 13379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:19.175527 13374 server_base.cc:1034] running on GCE node
I20260926 14:56:19.175853 13374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:19.176117 13374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:19.177251 13374 hybrid_clock.cc:648] HybridClock initialized: now 1790434579177238 us; error 26 us; skew 500 ppm
Sep 26 14:56:19 dist-test-slave-f9xb krb5kdc[13361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434579, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:19.179418 13374 init.cc:377] Logged in from keytab as oryx/127.7.37.254@KRBTEST.COM (short username oryx)
I20260926 14:56:19.179868 13374 webserver.cc:460] Webserver started at http://127.7.37.254:38455/ using document root <none> and password file <none>
I20260926 14:56:19.180083 13374 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:19.180130 13374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:19.180217 13374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:19.181075 13374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "0e48c6b89a3847f2835be52b19a380ff"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "a08a94e5c9922e984fb155b1bcec9eb0"
server_key_iv: "402f51c9fcd9704414a088fc881b830f"
server_key_version: "encryptionkey@0"
I20260926 14:56:19.181330 13374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "0e48c6b89a3847f2835be52b19a380ff"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "a08a94e5c9922e984fb155b1bcec9eb0"
server_key_iv: "402f51c9fcd9704414a088fc881b830f"
server_key_version: "encryptionkey@0"
I20260926 14:56:19.182410 13374 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:19.182984 13389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:19.183115 13374 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:19.183187 13374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "0e48c6b89a3847f2835be52b19a380ff"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "a08a94e5c9922e984fb155b1bcec9eb0"
server_key_iv: "402f51c9fcd9704414a088fc881b830f"
server_key_version: "encryptionkey@0"
I20260926 14:56:19.183249 13374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:19.190261 13374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:19.191824 13374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:19.191951 13374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:19.195317 13374 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:34241
I20260926 14:56:19.195343 13441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:34241 every 8 connection(s)
I20260926 14:56:19.195600 13374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:56:19.196455 13442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:19.196660 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 13374
I20260926 14:56:19.196786 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:56:19.196990 7319 external_mini_cluster.cc:1428] Setting key 8aa0becfe3b804b2659b7f9b96c6b49a
I20260926 14:56:19.199144 13442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff: Bootstrap starting.
I20260926 14:56:19.199666 13442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:19.199971 13442 log.cc:824] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:19.200624 13442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff: No bootstrap required, opened a new log
I20260926 14:56:19.202819 13442 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } }
I20260926 14:56:19.202957 13442 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:19.202992 13442 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e48c6b89a3847f2835be52b19a380ff, State: Initialized, Role: FOLLOWER
I20260926 14:56:19.203110 13442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } }
I20260926 14:56:19.203194 13442 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:56:19.203228 13442 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:56:19.203286 13442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [term 0 FOLLOWER]: Advancing to term 1
Sep 26 14:56:19 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM
I20260926 14:56:19.203943 13442 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } }
I20260926 14:56:19.204053 13442 leader_election.cc:302] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e48c6b89a3847f2835be52b19a380ff; no voters:
I20260926 14:56:19.204212 13442 leader_election.cc:288] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:56:19.204242 13447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:19.204439 13442 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:19.204696 13447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [term 1 LEADER]: Becoming Leader. State: Replica: 0e48c6b89a3847f2835be52b19a380ff, State: Running, Role: LEADER
I20260926 14:56:19.204807 13447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } }
I20260926 14:56:19.206058 13449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e48c6b89a3847f2835be52b19a380ff. Latest consensus state: current_term: 1 leader_uuid: "0e48c6b89a3847f2835be52b19a380ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } }
I20260926 14:56:19.206069 13448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e48c6b89a3847f2835be52b19a380ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } }
I20260926 14:56:19.206236 13449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:19.206243 13448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: This master's current role is: LEADER
W20260926 14:56:19.206562 13461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 14:56:19.206598 13461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 14:56:19.206709 13463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:56:19.206895 13463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:56:19.207412 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.197456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52278 (local address 127.7.37.254:34241)
0926 14:56:19.197598 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:19.197606 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:19.197630 (+ 24us) server_negotiation.cc:407] Connection header received
0926 14:56:19.197995 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:19.198000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:19.198045 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:19.198146 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:19.198924 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.199843 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.200825 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.201209 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.201286 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.203921 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:19.203924 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:19.203925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:19.203934 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:19.205452 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.205833 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.205837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.205837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.205938 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.206165 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.206166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.206167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.206449 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:19.206573 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:19.207093 (+ 520us) server_negotiation.cc:299] Negotiation successful
0926 14:56:19.207231 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":54,"threads_started":1}
I20260926 14:56:19.208339 13463 catalog_manager.cc:1347] Generated new cluster ID: dcfe92ec089b4fe7817234b268b008d2
I20260926 14:56:19.208397 13463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:56:19.226699 13463 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:56:19.227150 13463 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:19.232355 13463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff: Generated new TSK 0
I20260926 14:56:19.232508 13463 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.193.keytab.
Entry for principal oryx/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.193.keytab.
I20260926 14:56:19.257830 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:34241
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.193.keytab
--principal=oryx/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:19.355531 13468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:19.355700 13468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:19.355728 13468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:19.356840 13468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:19.356894 13468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:19.356928 13468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:56:19.358287 13468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.193.keytab
--principal=oryx/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:34241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:19.358493 13468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:19.358678 13468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:19.363147 13474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:19.363153 13476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:19.363160 13473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:19.363305 13468 server_base.cc:1034] running on GCE node
I20260926 14:56:19.363533 13468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:19.363716 13468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:19.364866 13468 hybrid_clock.cc:648] HybridClock initialized: now 1790434579364855 us; error 26 us; skew 500 ppm
Sep 26 14:56:19 dist-test-slave-f9xb krb5kdc[13361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434579, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:19.367146 13468 init.cc:377] Logged in from keytab as oryx/127.7.37.193@KRBTEST.COM (short username oryx)
I20260926 14:56:19.367518 13468 webserver.cc:460] Webserver started at http://127.7.37.193:42999/ using document root <none> and password file <none>
I20260926 14:56:19.367705 13468 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:19.367780 13468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:19.367904 13468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:19.368803 13468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "8ae81595f9464cadabc6fc7748af9c70"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "1e45f2b6ec6f64c90f0d109e43289a42"
server_key_iv: "f4f7c04d519a5f18990da2e2bc2b8a0e"
server_key_version: "encryptionkey@0"
I20260926 14:56:19.369086 13468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "8ae81595f9464cadabc6fc7748af9c70"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "1e45f2b6ec6f64c90f0d109e43289a42"
server_key_iv: "f4f7c04d519a5f18990da2e2bc2b8a0e"
server_key_version: "encryptionkey@0"
I20260926 14:56:19.370163 13468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:19.370757 13483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:19.370891 13468 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:19.370957 13468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "8ae81595f9464cadabc6fc7748af9c70"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "1e45f2b6ec6f64c90f0d109e43289a42"
server_key_iv: "f4f7c04d519a5f18990da2e2bc2b8a0e"
server_key_version: "encryptionkey@0"
I20260926 14:56:19.371027 13468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:19.389498 13468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:19.391006 13468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:19.391129 13468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:19.391291 13468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:19.391525 13468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:19.391553 13468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:19.391571 13468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:19.391593 13468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:19.396836 13468 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:42405
I20260926 14:56:19.396881 13596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:42405 every 8 connection(s)
I20260926 14:56:19.397109 13468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:56:19.401940 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 13468
Sep 26 14:56:19 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434579, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM
I20260926 14:56:19.402021 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:56:19.402177 7319 external_mini_cluster.cc:1428] Setting key 346fd89cc6454ee325273ab46902b068
I20260926 14:56:19.404778 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.397642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:57489 (local address 127.7.37.254:34241)
0926 14:56:19.397678 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:19.397680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:19.397854 (+ 174us) server_negotiation.cc:407] Connection header received
0926 14:56:19.398245 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:19.398246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:19.398254 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:19.398304 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:19.399059 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.399525 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.400393 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.400580 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.400622 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.402411 (+ 1789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:19.402413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:19.402413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:19.402421 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:19.403588 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.403971 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.403972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.403973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.404021 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.404223 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.404224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.404224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.404332 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:19.404384 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:19.404618 (+ 234us) server_negotiation.cc:299] Negotiation successful
0926 14:56:19.404694 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260926 14:56:19.404906 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.397623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.193:57489)
0926 14:56:19.397760 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:19.397794 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:19.398176 (+ 382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:19.398349 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:19.398351 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:19.398826 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:19.398994 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:19.398998 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.399583 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.399583 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:19.400339 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:19.400340 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.400630 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.400631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:19.400717 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.401115 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:19.401123 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:19.402329 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:19.403657 (+ 1328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:19.403663 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:19.403666 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:19.403908 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:19.404074 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:19.404074 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:19.404075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:19.404171 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:19.404390 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:19.404396 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:19.404512 (+ 116us) client_negotiation.cc:769] Sending connection context
0926 14:56:19.404592 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 14:56:19.404722 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260926 14:56:19.405321 13597 heartbeater.cc:347] Connected to a master server at 127.7.37.254:34241
I20260926 14:56:19.405422 13597 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:19.405578 13597 heartbeater.cc:511] Master 127.7.37.254:34241 requested a full tablet report, sending...
I20260926 14:56:19.406051 13406 ts_manager.cc:194] Registered new tserver with Master: 8ae81595f9464cadabc6fc7748af9c70 (127.7.37.193:42405)
I20260926 14:56:19.406895 13406 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:57489
WARNING: no policy specified for oryx/127.7.37.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.37.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.194.keytab.
Entry for principal oryx/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.194.keytab.
I20260926 14:56:19.424211 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:34241
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.194.keytab
--principal=oryx/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:19.500351 13602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:19.500522 13602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:19.500542 13602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:19.501581 13602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:19.501612 13602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:19.501636 13602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:56:19.502955 13602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.194.keytab
--principal=oryx/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:34241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:19.503105 13602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:19.503259 13602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:19.507714 13608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:19.507730 13607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:19.507783 13602 server_base.cc:1034] running on GCE node
W20260926 14:56:19.507714 13610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:19.508112 13602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:19.508342 13602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:19.509477 13602 hybrid_clock.cc:648] HybridClock initialized: now 1790434579509461 us; error 30 us; skew 500 ppm
Sep 26 14:56:19 dist-test-slave-f9xb krb5kdc[13361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434579, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:19.511677 13602 init.cc:377] Logged in from keytab as oryx/127.7.37.194@KRBTEST.COM (short username oryx)
I20260926 14:56:19.512101 13602 webserver.cc:460] Webserver started at http://127.7.37.194:44645/ using document root <none> and password file <none>
I20260926 14:56:19.512280 13602 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:19.512326 13602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:19.512435 13602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:19.513334 13602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "1445225627d3420ea69437d8763866b7"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "687f98c735481d4485c434d3060e7f1e"
server_key_iv: "b974bd24be4ba9842f8d36b9b78583d8"
server_key_version: "encryptionkey@0"
I20260926 14:56:19.513636 13602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "1445225627d3420ea69437d8763866b7"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "687f98c735481d4485c434d3060e7f1e"
server_key_iv: "b974bd24be4ba9842f8d36b9b78583d8"
server_key_version: "encryptionkey@0"
I20260926 14:56:19.514815 13602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 14:56:19.515468 13617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:19.515604 13602 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:19.515681 13602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "1445225627d3420ea69437d8763866b7"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "687f98c735481d4485c434d3060e7f1e"
server_key_iv: "b974bd24be4ba9842f8d36b9b78583d8"
server_key_version: "encryptionkey@0"
I20260926 14:56:19.515779 13602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:19.525655 13602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:19.527204 13602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:19.527328 13602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:19.527503 13602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:19.527779 13602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:19.527822 13602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:19.527856 13602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:19.527884 13602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:19.533427 13602 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:42005
I20260926 14:56:19.533458 13730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:42005 every 8 connection(s)
I20260926 14:56:19.533712 13602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:56:19.537591 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 13602
I20260926 14:56:19.537686 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:56:19.537863 7319 external_mini_cluster.cc:1428] Setting key 4255b2ed1f62376eafee1ef92c245534
Sep 26 14:56:19 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434579, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM
I20260926 14:56:19.541682 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.534304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:36211 (local address 127.7.37.254:34241)
0926 14:56:19.534353 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:19.534355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:19.534524 (+ 169us) server_negotiation.cc:407] Connection header received
0926 14:56:19.534935 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:19.534936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:19.534947 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:19.534995 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:19.535793 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.536269 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.537160 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.537352 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.537412 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.539255 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:19.539256 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:19.539257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:19.539263 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:19.540541 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.540891 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.540892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.540892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.540942 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.541143 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.541144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.541144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.541241 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:19.541300 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:19.541524 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 14:56:19.541599 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 14:56:19.541849 13733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.534293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.194:36211)
0926 14:56:19.534420 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:19.534448 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:19.534836 (+ 388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:19.535053 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:19.535057 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:19.535531 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:19.535690 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:19.535694 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.536342 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.536343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:19.537084 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:19.537086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.537418 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.537419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:19.537531 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.537959 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:19.537967 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:19.539180 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:19.540612 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:19.540617 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:19.540619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:19.540826 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:19.540996 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:19.540996 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:19.540997 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:19.541091 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:19.541292 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:19.541297 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:19.541418 (+ 121us) client_negotiation.cc:769] Sending connection context
0926 14:56:19.541499 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 14:56:19.541630 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":70,"threads_started":1}
I20260926 14:56:19.542198 13731 heartbeater.cc:347] Connected to a master server at 127.7.37.254:34241
I20260926 14:56:19.542271 13731 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:19.542410 13731 heartbeater.cc:511] Master 127.7.37.254:34241 requested a full tablet report, sending...
I20260926 14:56:19.542712 13406 ts_manager.cc:194] Registered new tserver with Master: 1445225627d3420ea69437d8763866b7 (127.7.37.194:42005)
I20260926 14:56:19.543174 13406 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:36211
WARNING: no policy specified for oryx/127.7.37.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.37.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.195.keytab.
Entry for principal oryx/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.195.keytab.
I20260926 14:56:19.559788 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:34241
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.195.keytab
--principal=oryx/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:19.634332 13736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:19.634500 13736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:19.634521 13736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:19.635586 13736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:19.635622 13736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:19.635648 13736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:56:19.637006 13736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.195.keytab
--principal=oryx/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:34241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:19.637181 13736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:19.637377 13736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:19.641803 13741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:19.641860 13736 server_base.cc:1034] running on GCE node
W20260926 14:56:19.641830 13742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:19.641968 13744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:19.642170 13736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:19.642347 13736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:19.643484 13736 hybrid_clock.cc:648] HybridClock initialized: now 1790434579643470 us; error 26 us; skew 500 ppm
Sep 26 14:56:19 dist-test-slave-f9xb krb5kdc[13361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434579, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:19.645615 13736 init.cc:377] Logged in from keytab as oryx/127.7.37.195@KRBTEST.COM (short username oryx)
I20260926 14:56:19.645974 13736 webserver.cc:460] Webserver started at http://127.7.37.195:38979/ using document root <none> and password file <none>
I20260926 14:56:19.646155 13736 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:19.646198 13736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:19.646311 13736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:19.647195 13736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "356c4e7736584de99be4f7f0a33b2db1"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "540a4d0ce9ed1d709e4e0ad5add96c8e"
server_key_iv: "9bbb6ba93dea14000c1cb530832490cc"
server_key_version: "encryptionkey@0"
I20260926 14:56:19.647478 13736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "356c4e7736584de99be4f7f0a33b2db1"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "540a4d0ce9ed1d709e4e0ad5add96c8e"
server_key_iv: "9bbb6ba93dea14000c1cb530832490cc"
server_key_version: "encryptionkey@0"
I20260926 14:56:19.648626 13736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 14:56:19.649228 13751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:19.649367 13736 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:56:19.649443 13736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "356c4e7736584de99be4f7f0a33b2db1"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "540a4d0ce9ed1d709e4e0ad5add96c8e"
server_key_iv: "9bbb6ba93dea14000c1cb530832490cc"
server_key_version: "encryptionkey@0"
I20260926 14:56:19.649503 13736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:19.664808 13736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:19.666332 13736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:19.666448 13736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:19.666612 13736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:19.666849 13736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:19.666886 13736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:19.666911 13736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:19.666927 13736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:19.671933 13736 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:42945
I20260926 14:56:19.671990 13864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:42945 every 8 connection(s)
I20260926 14:56:19.672211 13736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:56:19.673154 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 13736
I20260926 14:56:19.673256 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:56:19.673408 7319 external_mini_cluster.cc:1428] Setting key 7e206726c3c7375ab46420ff87f346a4
Sep 26 14:56:19 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434579, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM
I20260926 14:56:19.680024 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.672755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:48567 (local address 127.7.37.254:34241)
0926 14:56:19.672817 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:19.672819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:19.672941 (+ 122us) server_negotiation.cc:407] Connection header received
0926 14:56:19.673490 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:19.673491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:19.673502 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:19.673560 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:19.674511 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.675013 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.675946 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.676124 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.676166 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.677902 (+ 1736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:19.677903 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:19.677903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:19.677909 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:19.679068 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.679350 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.679350 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.679350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.679411 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.679569 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.679570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.679570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.679654 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:19.679694 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:19.679914 (+ 220us) server_negotiation.cc:299] Negotiation successful
0926 14:56:19.679967 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 14:56:19.680148 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.672721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.195:48567)
0926 14:56:19.672849 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:19.672877 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:19.673390 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:19.673646 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:19.673649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:19.674253 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:19.674441 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:19.674446 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.675073 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.675073 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:19.675887 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:19.675890 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.676173 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.676173 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:19.676261 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.676634 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:19.676642 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:19.677823 (+ 1181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:19.679123 (+ 1300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:19.679126 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:19.679128 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:19.679302 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:19.679454 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:19.679455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:19.679455 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:19.679530 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:19.679692 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:19.679695 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:19.679826 (+ 131us) client_negotiation.cc:769] Sending connection context
0926 14:56:19.679899 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 14:56:19.679998 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":61,"threads_started":1}
I20260926 14:56:19.680481 13865 heartbeater.cc:347] Connected to a master server at 127.7.37.254:34241
I20260926 14:56:19.680546 13865 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:19.680656 13865 heartbeater.cc:511] Master 127.7.37.254:34241 requested a full tablet report, sending...
I20260926 14:56:19.680928 13406 ts_manager.cc:194] Registered new tserver with Master: 356c4e7736584de99be4f7f0a33b2db1 (127.7.37.195:42945)
I20260926 14:56:19.681242 13406 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:48567
I20260926 14:56:19.684854 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.37.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.37.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.37.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.253.keytab.
Entry for principal oryx/127.7.37.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.253.keytab.
I20260926 14:56:19.705935 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.253:33917
--webserver_interface=127.7.37.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.253.keytab
--principal=oryx/127.7.37.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:34241,127.7.37.253:33917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:19.780697 13870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:19.780876 13870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:19.780905 13870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:19.781975 13870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:19.782011 13870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:19.782029 13870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:19.782044 13870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:19.782060 13870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:56:19.783442 13870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:34241,127.7.37.253:33917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.253.keytab
--principal=oryx/127.7.37.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.253:33917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.37.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
W20260926 14:56:19.783538 13870 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.37.254:34241,127.7.37.253:33917'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260926 14:56:19.783674 13870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:19.783867 13870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:19.788132 13875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:19.788139 13878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:19.788178 13870 server_base.cc:1034] running on GCE node
W20260926 14:56:19.788149 13876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:19.788503 13870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:19.788744 13870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:19.789871 13870 hybrid_clock.cc:648] HybridClock initialized: now 1790434579789858 us; error 25 us; skew 500 ppm
Sep 26 14:56:19 dist-test-slave-f9xb krb5kdc[13361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434579, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:19.792029 13870 init.cc:377] Logged in from keytab as oryx/127.7.37.253@KRBTEST.COM (short username oryx)
I20260926 14:56:19.792428 13870 webserver.cc:460] Webserver started at http://127.7.37.253:33091/ using document root <none> and password file <none>
I20260926 14:56:19.792604 13870 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:19.792662 13870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:19.792788 13870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:19.793922 13870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/data/instance:
uuid: "e3a46c25f46f453d9991a98b0f3ecff8"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "46a43421eec440f5cf5858e0ab24c715"
server_key_iv: "4261362bfd1f0eb244eabe1be5095bde"
server_key_version: "encryptionkey@0"
I20260926 14:56:19.794214 13870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/wal/instance:
uuid: "e3a46c25f46f453d9991a98b0f3ecff8"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "46a43421eec440f5cf5858e0ab24c715"
server_key_iv: "4261362bfd1f0eb244eabe1be5095bde"
server_key_version: "encryptionkey@0"
I20260926 14:56:19.795326 13870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 14:56:19.796047 13885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:19.796181 13870 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:19.796264 13870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/wal
uuid: "e3a46c25f46f453d9991a98b0f3ecff8"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "46a43421eec440f5cf5858e0ab24c715"
server_key_iv: "4261362bfd1f0eb244eabe1be5095bde"
server_key_version: "encryptionkey@0"
I20260926 14:56:19.796324 13870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:19.814719 13870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:19.816552 13870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:19.816665 13870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:19.820098 13870 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.253:33917
I20260926 14:56:19.820124 13937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.253:33917 every 8 connection(s)
I20260926 14:56:19.820393 13870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/data/info.pb
I20260926 14:56:19.821264 13938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:19.822647 13938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 14:56:19 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434579, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.253@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM
I20260926 14:56:19.829550 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 13870
I20260926 14:56:19.830163 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.823258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52290 (local address 127.7.37.254:34241)
0926 14:56:19.823309 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:19.823310 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:19.823491 (+ 181us) server_negotiation.cc:407] Connection header received
0926 14:56:19.823918 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:19.823919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:19.823929 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:19.823979 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:19.824461 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.824924 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.825809 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.825996 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.826048 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.827830 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:19.827832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:19.827832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:19.827839 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:19.828923 (+ 1084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.829230 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.829231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.829231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.829281 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.829465 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.829466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.829466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.829662 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:19.829745 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:19.829990 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 14:56:19.830076 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 14:56:19.830291 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.823245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:52290)
0926 14:56:19.823383 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:19.823412 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:19.823845 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:19.824023 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:19.824027 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:19.824229 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:19.824395 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:19.824398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.824977 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.824978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:19.825736 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:19.825737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.826053 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.826053 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:19.826138 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.826514 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:19.826521 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:19.827713 (+ 1192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:19.828989 (+ 1276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:19.828994 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:19.828995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:19.829174 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:19.829326 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:19.829326 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:19.829327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:19.829402 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:19.829732 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:19.829737 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:19.829850 (+ 113us) client_negotiation.cc:769] Sending connection context
0926 14:56:19.829951 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 14:56:19.830095 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260926 14:56:19.830852 13938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 14:56:19 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434579, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.253@KRBTEST.COM for oryx/127.7.37.253@KRBTEST.COM
Sep 26 14:56:19 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.37.253@KRBTEST.COM
I20260926 14:56:19.837492 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.831064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:33917 (local address 127.0.0.1:41358)
0926 14:56:19.831100 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:19.831109 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:19.831142 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:19.831324 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:19.831325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:19.831424 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:19.831467 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:19.831468 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.832347 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.832348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:19.832806 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:19.832807 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.833212 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.833213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:19.833268 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.833625 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:19.833629 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:19.835092 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:19.836612 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:19.836612 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:19.836613 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:19.836748 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:19.836928 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:19.836929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:19.836929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:19.836989 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:19.837270 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:19.837271 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:19.837304 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 14:56:19.837355 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 14:56:19.837440 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260926 14:56:19.837602 13943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.831115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41358 (local address 127.7.37.253:33917)
0926 14:56:19.831232 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:19.831233 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:19.831238 (+ 5us) server_negotiation.cc:407] Connection header received
0926 14:56:19.831252 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:19.831253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:19.831259 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:19.831315 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:19.831525 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.832289 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.832869 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.833146 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.833197 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.835155 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:19.835161 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:19.835161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:19.835167 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:19.836548 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.836806 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.836809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.836809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.836863 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.837044 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.837045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.837045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.837205 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:19.837281 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:19.837395 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 14:56:19.837491 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":104,"thread_start_us":83,"threads_started":1}
I20260926 14:56:19.838790 13938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: Bootstrap starting.
I20260926 14:56:19.839306 13942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.830062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41350 (local address 127.7.37.253:33917)
0926 14:56:19.830178 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:19.830181 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:19.830195 (+ 14us) server_negotiation.cc:407] Connection header received
0926 14:56:19.830213 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:19.830215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:19.830266 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:19.830340 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:19.831261 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.832275 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.833494 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.833780 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.833843 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.836226 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:19.836227 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:19.836227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:19.836233 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:19.838364 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.838641 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.838642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.838642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.838690 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.838839 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.838839 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.838839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.838951 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:19.839069 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:19.839132 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 14:56:19.839195 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
I20260926 14:56:19.839345 13938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:19.839648 13938 log.cc:824] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:19.840272 13938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: No bootstrap required, opened a new log
I20260926 14:56:19.841475 13938 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } }
I20260926 14:56:19.841578 13938 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:19.841606 13938 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3a46c25f46f453d9991a98b0f3ecff8, State: Initialized, Role: FOLLOWER
I20260926 14:56:19.841688 13938 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } }
I20260926 14:56:19.841904 13938 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:19.841881 13945 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } } }
I20260926 14:56:19.841948 13945 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 14:56:19.843402 13956 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 14:56:19.843452 13956 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 14:56:19.844622 13870 master_runner.cc:428] Detected that this master e3a46c25f46f453d9991a98b0f3ecff8 is joining an existing cluster
I20260926 14:56:19.844672 13870 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.37.253:33917
I20260926 14:56:19.851158 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.845715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52292 (local address 127.7.37.254:34241)
0926 14:56:19.845789 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:19.845791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:19.845997 (+ 206us) server_negotiation.cc:407] Connection header received
0926 14:56:19.846081 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:19.846082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:19.846091 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:19.846133 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:19.846454 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.847145 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.848030 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.848299 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.848365 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.849119 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:19.849120 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:19.849120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:19.849126 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:19.850195 (+ 1069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.850528 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.850529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.850530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.850578 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.850725 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.850726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.850726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.850831 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:19.850881 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:19.851003 (+ 122us) server_negotiation.cc:299] Negotiation successful
0926 14:56:19.851075 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 14:56:19.851176 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.845679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:52292)
0926 14:56:19.845941 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:19.845951 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:19.846004 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:19.846136 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:19.846137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:19.846343 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:19.846392 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:19.846394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.847219 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.847219 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:19.847966 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:19.847968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.848368 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.848369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:19.848431 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.848715 (+ 284us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:19.848719 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:19.849056 (+ 337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:19.850261 (+ 1205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:19.850262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:19.850262 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:19.850472 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:19.850626 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:19.850627 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:19.850627 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:19.850679 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:19.850881 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:19.850881 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:19.850918 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 14:56:19.850978 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 14:56:19.851072 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":2,"thread_start_us":61,"threads_started":1}
I20260926 14:56:19.852135 13942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.845757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41366 (local address 127.7.37.253:33917)
0926 14:56:19.845813 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:19.845827 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:19.845877 (+ 50us) server_negotiation.cc:407] Connection header received
0926 14:56:19.845981 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:19.845982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:19.845991 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:19.846038 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:19.846404 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.846863 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.847686 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.848012 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.848085 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.848999 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:19.849000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:19.849000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:19.849006 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:19.851030 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.851378 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.851379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.851379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.851422 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.851580 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.851580 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.851581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.851677 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:19.851709 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:19.851936 (+ 227us) server_negotiation.cc:299] Negotiation successful
0926 14:56:19.852045 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":44}
I20260926 14:56:19.852344 13962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.845679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:33917 (local address 127.0.0.1:41366)
0926 14:56:19.845818 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:19.845827 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:19.845897 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:19.846043 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:19.846043 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:19.846243 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:19.846322 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:19.846324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.846924 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.846925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:19.847608 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:19.847610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.848084 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.848085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:19.848143 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.848553 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:19.848556 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:19.848921 (+ 365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:19.851116 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:19.851116 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:19.851117 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:19.851300 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:19.851479 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:19.851480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:19.851480 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:19.851525 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:19.851794 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:19.851794 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:19.851830 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 14:56:19.851897 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 14:56:19.851988 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":94,"threads_started":1}
Sep 26 14:56:19 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434579, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.254@KRBTEST.COM for oryx/127.7.37.253@KRBTEST.COM
I20260926 14:56:19.859372 13942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.853222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41380 (local address 127.7.37.253:33917)
0926 14:56:19.853262 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:19.853263 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:19.853351 (+ 88us) server_negotiation.cc:407] Connection header received
0926 14:56:19.853428 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:19.853429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:19.853435 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:19.853475 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:19.853888 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.854288 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.855284 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.855446 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.855482 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.857362 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:19.857363 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:19.857363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:19.857369 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:19.858449 (+ 1080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.858812 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.858812 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.858812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.858858 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.859017 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.859017 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.859018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.859093 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:19.859131 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:19.859279 (+ 148us) server_negotiation.cc:299] Negotiation successful
0926 14:56:19.859326 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 14:56:19.859465 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.853150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:33917 (local address 127.0.0.1:41380)
0926 14:56:19.853254 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:19.853289 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:19.853364 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:19.853481 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:19.853484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:19.853777 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:19.853825 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:19.853827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.854348 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.854349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:19.855220 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:19.855222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.855498 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.855499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:19.855570 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.855972 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:19.855976 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:19.857274 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:19.858514 (+ 1240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:19.858519 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:19.858521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:19.858751 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:19.858907 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:19.858908 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:19.858908 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:19.858967 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:19.859138 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:19.859143 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:19.859186 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 14:56:19.859257 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 14:56:19.859370 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":59,"threads_started":1}
I20260926 14:56:19.859964 13406 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.37.253:33917: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8"
member_type: NON_VOTER
last_known_addr {
host: "127.7.37.253"
port: 33917
}
attrs {
promote: true
}
}
dest_uuid: "0e48c6b89a3847f2835be52b19a380ff"
cas_config_opid_index: -1
I20260926 14:56:19.860227 13406 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: NON_VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: true } }
I20260926 14:56:19.860646 13965 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER e3a46c25f46f453d9991a98b0f3ecff8 (127.7.37.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: NON_VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: true } } }
I20260926 14:56:19.860890 13912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:19.861485 13912 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 1 FOLLOWER]: Refusing update from remote peer 0e48c6b89a3847f2835be52b19a380ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260926 14:56:19.861557 13965 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.37.253:33917
I20260926 14:56:19.861635 13965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0e48c6b89a3847f2835be52b19a380ff" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: NON_VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: true } } }
I20260926 14:56:19.861697 13965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:19.861780 13965 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: NON_VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:19.861833 13870 master.cc:537] Master@127.7.37.253:33917 shutting down...
W20260926 14:56:19.863080 13390 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260926 14:56:19.863461 13390 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff -> Peer e3a46c25f46f453d9991a98b0f3ecff8 (127.7.37.253:33917): Couldn't send request to peer e3a46c25f46f453d9991a98b0f3ecff8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 14:56:19.864251 13870 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:19.864329 13870 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:19.864373 13870 tablet_replica.cc:330] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: stopping tablet replica
I20260926 14:56:19.876601 13870 master.cc:552] Master@127.7.37.253:33917 shutdown complete.
I20260926 14:56:19.876644 13870 master_runner.cc:305] Clearing existing system tablet
I20260926 14:56:19.876754 13870 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:19.876796 13870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:19.877385 13870 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:19.877812 13970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:19.877939 13870 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:19.877990 13870 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:19.878008 13870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/wal
uuid: "e3a46c25f46f453d9991a98b0f3ecff8"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "46a43421eec440f5cf5858e0ab24c715"
server_key_iv: "4261362bfd1f0eb244eabe1be5095bde"
server_key_version: "encryptionkey@0"
I20260926 14:56:19.878490 13870 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:19.879575 13870 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 14:56:19.879622 13870 log.cc:1196] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260926 14:56:19.879799 13870 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: Deleting consensus metadata
I20260926 14:56:19.879904 13870 master_runner.cc:315] Copying system tablet from 127.7.37.254:34241
I20260926 14:56:19.880362 13870 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: tablet copy: Beginning tablet copy session from remote peer at address 127.7.37.254:34241
I20260926 14:56:19.885200 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.880619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52296 (local address 127.7.37.254:34241)
0926 14:56:19.880649 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:19.880651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:19.880740 (+ 89us) server_negotiation.cc:407] Connection header received
0926 14:56:19.880791 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:19.880792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:19.880801 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:19.880857 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:19.881100 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.881538 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.882201 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.882391 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.882439 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.883156 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:19.883157 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:19.883158 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:19.883163 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:19.884312 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.884586 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.884587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.884587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.884635 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.884784 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.884785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.884785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.884885 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:19.884936 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:19.885059 (+ 123us) server_negotiation.cc:299] Negotiation successful
0926 14:56:19.885117 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260926 14:56:19.885217 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.880579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:52296)
0926 14:56:19.880677 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:19.880686 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:19.880740 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:19.880862 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:19.880862 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:19.880992 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:19.881041 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:19.881042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.881599 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.881599 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:19.882135 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:19.882136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.882442 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.882443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:19.882505 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.882843 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:19.882847 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:19.883082 (+ 235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:19.884382 (+ 1300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:19.884382 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:19.884383 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:19.884530 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:19.884685 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:19.884685 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:19.884685 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:19.884736 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:19.884935 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:19.884935 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:19.884973 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 14:56:19.885032 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 14:56:19.885118 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":61,"threads_started":1}
I20260926 14:56:19.885632 13426 tablet_copy_service.cc:140] P 0e48c6b89a3847f2835be52b19a380ff: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e3a46c25f46f453d9991a98b0f3ecff8 ({username='oryx', principal='oryx/127.7.37.253@KRBTEST.COM'} at 127.0.0.1:52296)
I20260926 14:56:19.885711 13426 tablet_copy_service.cc:161] P 0e48c6b89a3847f2835be52b19a380ff: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e3a46c25f46f453d9991a98b0f3ecff8 at {username='oryx', principal='oryx/127.7.37.253@KRBTEST.COM'} at 127.0.0.1:52296: session id = e3a46c25f46f453d9991a98b0f3ecff8-00000000000000000000000000000000
I20260926 14:56:19.886416 13426 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 14:56:19.886904 13870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 14:56:19.887594 13870 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: tablet copy: Starting download of 0 data blocks...
I20260926 14:56:19.887705 13870 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: tablet copy: Starting download of 1 WAL segments...
I20260926 14:56:19.888441 13870 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 14:56:19.889083 13426 tablet_copy_service.cc:342] P 0e48c6b89a3847f2835be52b19a380ff: Request end of tablet copy session e3a46c25f46f453d9991a98b0f3ecff8-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.37.253@KRBTEST.COM'} at 127.0.0.1:52296
I20260926 14:56:19.889125 13426 tablet_copy_service.cc:433] P 0e48c6b89a3847f2835be52b19a380ff: ending tablet copy session e3a46c25f46f453d9991a98b0f3ecff8-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e3a46c25f46f453d9991a98b0f3ecff8
I20260926 14:56:19.891086 13870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:19.892077 13980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:19.892117 13977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:19.892215 13870 server_base.cc:1034] running on GCE node
W20260926 14:56:19.892395 13978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:19.892542 13870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:19.892655 13870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:19.893761 13870 hybrid_clock.cc:648] HybridClock initialized: now 1790434579893755 us; error 21 us; skew 500 ppm
Sep 26 14:56:19 dist-test-slave-f9xb krb5kdc[13361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434579, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:19.894783 13870 init.cc:377] Logged in from keytab as oryx/127.7.37.253@KRBTEST.COM (short username oryx)
I20260926 14:56:19.894999 13870 webserver.cc:460] Webserver started at http://127.7.37.253:36311/ using document root <none> and password file <none>
I20260926 14:56:19.895102 13870 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:19.895131 13870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:19.895614 13870 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:19.896095 13987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:19.896212 13870 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:19.896252 13870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/wal
uuid: "e3a46c25f46f453d9991a98b0f3ecff8"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "46a43421eec440f5cf5858e0ab24c715"
server_key_iv: "4261362bfd1f0eb244eabe1be5095bde"
server_key_version: "encryptionkey@0"
I20260926 14:56:19.896296 13870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:19.930706 13870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:19.932166 13870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:19.934777 13870 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.253:33917
I20260926 14:56:19.934803 14039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.253:33917 every 8 connection(s)
I20260926 14:56:19.935047 13870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-1/data/info.pb
I20260926 14:56:19.935619 14040 sys_catalog.cc:263] Verifying existing consensus state
I20260926 14:56:19.936012 14040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: Bootstrap starting.
I20260926 14:56:19.937888 14040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 14:56:19.938047 14040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: Bootstrap complete.
I20260926 14:56:19.938213 14040 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: NON_VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: true } }
I20260926 14:56:19.938278 14040 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e3a46c25f46f453d9991a98b0f3ecff8, State: Initialized, Role: LEARNER
I20260926 14:56:19.938303 14040 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: NON_VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: true } }
I20260926 14:56:19.938449 14040 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:19.938462 14043 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: NON_VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: true } } }
I20260926 14:56:19.938557 14043 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [sys.catalog]: This master's current role is: LEARNER
I20260926 14:56:19.939648 14054 catalog_manager.cc:1259] Loaded cluster ID: dcfe92ec089b4fe7817234b268b008d2
I20260926 14:56:19.939690 14054 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: loading cluster ID for follower catalog manager: success
I20260926 14:56:19.940694 14054 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: acquiring CA information for follower catalog manager: success
I20260926 14:56:19.940989 14054 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 14:56:19.946655 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:19.940570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41398 (local address 127.7.37.253:33917)
0926 14:56:19.940655 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:19.940657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:19.940690 (+ 33us) server_negotiation.cc:407] Connection header received
0926 14:56:19.940810 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:19.940811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:19.940822 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:19.940894 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:19.941485 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.942083 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.943028 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:19.943211 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:19.943257 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:19.944471 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:19.944472 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:19.944472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:19.944478 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:19.945673 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.946026 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.946026 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.946027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.946068 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:19.946244 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:19.946245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:19.946245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:19.946334 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:19.946408 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:19.946537 (+ 129us) server_negotiation.cc:299] Negotiation successful
0926 14:56:19.946589 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70,"thread_start_us":45,"threads_started":1}
WARNING: no policy specified for oryx/127.7.37.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.37.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.37.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.252.keytab.
Entry for principal oryx/127.7.37.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.252.keytab.
I20260926 14:56:19.967603 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.252:39947
--webserver_interface=127.7.37.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.252.keytab
--principal=oryx/127.7.37.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:34241,127.7.37.253:33917,127.7.37.252:39947
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:20.043466 14058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:20.043632 14058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:20.043651 14058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:20.044757 14058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:20.044795 14058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:20.044808 14058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:20.044821 14058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:20.044850 14058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:56:20.046278 14058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:34241,127.7.37.253:33917,127.7.37.252:39947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.252.keytab
--principal=oryx/127.7.37.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.252:39947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.37.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:20.046499 14058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:20.046686 14058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:20.050771 14066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:20.050953 14064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:20.050932 14063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:20.051020 14058 server_base.cc:1034] running on GCE node
I20260926 14:56:20.051277 14058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:20.051530 14058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:20.052654 14058 hybrid_clock.cc:648] HybridClock initialized: now 1790434580052641 us; error 28 us; skew 500 ppm
Sep 26 14:56:20 dist-test-slave-f9xb krb5kdc[13361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434580, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:20.054702 14058 init.cc:377] Logged in from keytab as oryx/127.7.37.252@KRBTEST.COM (short username oryx)
I20260926 14:56:20.055047 14058 webserver.cc:460] Webserver started at http://127.7.37.252:34227/ using document root <none> and password file <none>
I20260926 14:56:20.055227 14058 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:20.055269 14058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:20.055387 14058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:20.056314 14058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/data/instance:
uuid: "8f21b88955a146c9a566f880dafac879"
format_stamp: "Formatted at 2026-09-26 14:56:20 on dist-test-slave-f9xb"
server_key: "8b5dc78044bb4a86e0465b7a2535441b"
server_key_iv: "daa355e115c88b371be2f74d9d908720"
server_key_version: "encryptionkey@0"
I20260926 14:56:20.056597 14058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/wal/instance:
uuid: "8f21b88955a146c9a566f880dafac879"
format_stamp: "Formatted at 2026-09-26 14:56:20 on dist-test-slave-f9xb"
server_key: "8b5dc78044bb4a86e0465b7a2535441b"
server_key_iv: "daa355e115c88b371be2f74d9d908720"
server_key_version: "encryptionkey@0"
I20260926 14:56:20.057706 14058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:20.058271 14073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:20.058403 14058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:20.058470 14058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/wal
uuid: "8f21b88955a146c9a566f880dafac879"
format_stamp: "Formatted at 2026-09-26 14:56:20 on dist-test-slave-f9xb"
server_key: "8b5dc78044bb4a86e0465b7a2535441b"
server_key_iv: "daa355e115c88b371be2f74d9d908720"
server_key_version: "encryptionkey@0"
I20260926 14:56:20.058526 14058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:20.072039 14058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:20.073519 14058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:20.073628 14058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:20.077015 14058 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.252:39947
I20260926 14:56:20.077044 14125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.252:39947 every 8 connection(s)
I20260926 14:56:20.077306 14058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/data/info.pb
I20260926 14:56:20.078065 14126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:20.079317 14126 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } has no permanent_uuid. Determining permanent_uuid...
I20260926 14:56:20.080904 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 14058
Sep 26 14:56:20 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434580, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.252@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM
Sep 26 14:56:20 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.37.252@KRBTEST.COM
I20260926 14:56:20.087278 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.080007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52308 (local address 127.7.37.254:34241)
0926 14:56:20.080063 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.080066 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.080097 (+ 31us) server_negotiation.cc:407] Connection header received
0926 14:56:20.080547 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.080548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.080560 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.080608 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.081146 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.081661 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.082611 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.082802 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.082862 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.084724 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.084726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.084726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.084733 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.086007 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.086367 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.086368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.086368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.086425 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.086681 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.086682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.086682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.086777 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.086832 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.087126 (+ 294us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.087206 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 14:56:20.087386 14128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.079863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:52308)
0926 14:56:20.080000 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.080032 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.080453 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.080665 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.080669 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.080875 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:20.081068 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.081072 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.081733 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.081734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.082533 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.082535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.082864 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.082865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.082979 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.083380 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:20.083387 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:20.084653 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:20.086072 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.086079 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.086081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.086307 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.086486 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.086487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.086487 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.086622 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.086863 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:20.086869 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:20.087009 (+ 140us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.087089 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.087190 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260926 14:56:20.087841 14126 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } has no permanent_uuid. Determining permanent_uuid...
I20260926 14:56:20.089437 14129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.081293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54228 (local address 127.7.37.252:39947)
0926 14:56:20.081420 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.081424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.081484 (+ 60us) server_negotiation.cc:407] Connection header received
0926 14:56:20.081544 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.081546 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.081564 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.082011 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.082570 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.083217 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.083926 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.084187 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.084238 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.086527 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.086532 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.086532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.086537 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.088469 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.088794 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.088796 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.088796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.088841 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.089001 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.089001 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.089001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.089099 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.089202 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.089267 (+ 65us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.089346 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
Sep 26 14:56:20 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434580, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.252@KRBTEST.COM for oryx/127.7.37.253@KRBTEST.COM
I20260926 14:56:20.093421 14128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.088035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:33917 (local address 127.0.0.1:41400)
0926 14:56:20.088076 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.088086 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.088126 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.088237 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.088237 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.088421 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:20.088451 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.088452 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.089259 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.089260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.089939 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.089940 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.090237 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.090237 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.090272 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.090530 (+ 258us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:20.090533 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:20.091451 (+ 918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:20.092763 (+ 1312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.092764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.092764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.092895 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.093018 (+ 123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.093019 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.093019 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.093055 (+ 36us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.093200 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:20.093201 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:20.093235 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.093290 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.093361 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 14:56:20.093456 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.088088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41400 (local address 127.7.37.253:33917)
0926 14:56:20.088124 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.088126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.088134 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:20.088182 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.088183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.088193 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.088232 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.088516 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.089189 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.090001 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.090189 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.090235 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.091512 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.091513 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.091514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.091520 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.092707 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.092943 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.092943 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.092943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.092980 (+ 37us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.093085 (+ 105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.093085 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.093086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.093161 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.093200 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.093337 (+ 137us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.093399 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 14:56:20.093744 14126 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 14:56:20 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434580, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.252@KRBTEST.COM for oryx/127.7.37.252@KRBTEST.COM
I20260926 14:56:20.098711 14128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.093938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:39947 (local address 127.0.0.1:54244)
0926 14:56:20.094003 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.094012 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.094058 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.094155 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.094156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.094262 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:20.094282 (+ 20us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.094283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.094810 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.094810 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.095240 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.095241 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.095526 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.095527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.095569 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.095881 (+ 312us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:20.095885 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:20.096836 (+ 951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:20.097962 (+ 1126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.097962 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.097963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.098095 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.098232 (+ 137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.098233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.098233 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.098279 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.098456 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:20.098457 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:20.098489 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.098545 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.098623 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260926 14:56:20.098762 14129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.093975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54244 (local address 127.7.37.252:39947)
0926 14:56:20.094025 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.094026 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.094046 (+ 20us) server_negotiation.cc:407] Connection header received
0926 14:56:20.094104 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.094105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.094113 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.094154 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.094336 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.094735 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.095303 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.095476 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.095523 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.096899 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.096900 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.096900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.096906 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.097900 (+ 994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.098145 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.098146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.098147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.098186 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.098328 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.098329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.098330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.098409 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.098457 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.098568 (+ 111us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.098667 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":35}
I20260926 14:56:20.099665 14126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: Bootstrap starting.
I20260926 14:56:20.100361 14126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:20.100558 14126 log.cc:824] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:20.101105 14126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: No bootstrap required, opened a new log
I20260926 14:56:20.102515 14126 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } }
I20260926 14:56:20.102623 14126 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:20.102644 14126 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f21b88955a146c9a566f880dafac879, State: Initialized, Role: FOLLOWER
I20260926 14:56:20.102737 14126 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } }
I20260926 14:56:20.102977 14126 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:20.102980 14131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } } }
I20260926 14:56:20.103093 14131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 14:56:20.104324 14142 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 14:56:20.104365 14142 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 14:56:20.106376 14058 master_runner.cc:428] Detected that this master 8f21b88955a146c9a566f880dafac879 is joining an existing cluster
I20260926 14:56:20.106420 14058 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.37.252:39947
I20260926 14:56:20.113512 14149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.107283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:52314)
0926 14:56:20.107474 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.107482 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.107528 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.107673 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.107673 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.107849 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:20.107912 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.107913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.108805 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.108806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.109674 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.109676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.110158 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.110159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.110246 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.110617 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:20.110621 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:20.110907 (+ 286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:20.112690 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.112691 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.112691 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.112840 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.113011 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.113012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.113012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.113054 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.113270 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:20.113270 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:20.113326 (+ 56us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.113386 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.113451 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":21,"thread_start_us":74,"threads_started":1}
I20260926 14:56:20.113575 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.107327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52314 (local address 127.7.37.254:34241)
0926 14:56:20.107390 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.107392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.107530 (+ 138us) server_negotiation.cc:407] Connection header received
0926 14:56:20.107579 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.107580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.107589 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.107634 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.107998 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.108723 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.109762 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.110042 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.110077 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.111023 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.111024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.111024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.111031 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.112610 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.112894 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.112895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.112895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.112952 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.113101 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.113102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.113102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.113219 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.113280 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.113422 (+ 142us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.113489 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 26 14:56:20 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434579, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.254@KRBTEST.COM for oryx/127.7.37.252@KRBTEST.COM
I20260926 14:56:20.134568 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.115288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:39947 (local address 127.0.0.1:54260)
0926 14:56:20.115332 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.115341 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.115374 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.115668 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.115668 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.115822 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:20.115856 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.115857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.116723 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.116724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.117200 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.117201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.117666 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.117667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.117718 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.118071 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:20.118075 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:20.119163 (+ 1088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:20.133611 (+ 14448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.133612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.133612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.133805 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.133994 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.133995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.133995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.134046 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.134286 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:20.134287 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:20.134325 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.134381 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.134472 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260926 14:56:20.134986 14151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.115422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54260 (local address 127.7.37.252:39947)
0926 14:56:20.115547 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.115549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.115555 (+ 6us) server_negotiation.cc:407] Connection header received
0926 14:56:20.115583 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.115584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.115593 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.115663 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.115924 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.116659 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.117263 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.117597 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.117650 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.119228 (+ 1578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.119229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.119230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.119235 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.133520 (+ 14285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.133878 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.133879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.133879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.133927 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.134104 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.134104 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.134105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.134215 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.134723 (+ 508us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.134771 (+ 48us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.134890 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
I20260926 14:56:20.135327 13406 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.37.252:39947: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8f21b88955a146c9a566f880dafac879"
member_type: NON_VOTER
last_known_addr {
host: "127.7.37.252"
port: 39947
}
attrs {
promote: true
}
}
dest_uuid: "0e48c6b89a3847f2835be52b19a380ff"
cas_config_opid_index: 5
I20260926 14:56:20.135562 13406 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: NON_VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: true } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: NON_VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: true } }
I20260926 14:56:20.136010 14148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.107536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:39947 (local address 127.0.0.1:54258)
0926 14:56:20.107987 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.107995 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.108048 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.108176 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.108177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.108349 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:20.108403 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.108405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.109222 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.109223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.110023 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.110025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.110469 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.110469 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.110527 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.132439 (+ 21912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:20.132443 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:20.133248 (+ 805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:20.135070 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.135071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.135071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.135238 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.135407 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.135407 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.135408 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.135469 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.135722 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:20.135723 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:20.135775 (+ 52us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.135831 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.135914 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":65,"threads_started":1}
I20260926 14:56:20.136204 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.107692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41408 (local address 127.7.37.253:33917)
0926 14:56:20.107794 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.107796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.107804 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:20.107822 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.107823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.107831 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.107887 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.108170 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.108854 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.109783 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.110069 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.110101 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.132672 (+ 22571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.132673 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.132674 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.132680 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.134108 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.134460 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.134462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.134462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.134514 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.134937 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.134938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.134938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.135059 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.135097 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.136051 (+ 954us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.136120 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 14:56:20.136212 14129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.107732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54258 (local address 127.7.37.252:39947)
0926 14:56:20.107941 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.107942 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.108031 (+ 89us) server_negotiation.cc:407] Connection header received
0926 14:56:20.108103 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.108104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.108110 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.108133 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.108475 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.109142 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.110106 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.110370 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.110411 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.133566 (+ 23155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.133567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.133568 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.133574 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.135007 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.135295 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.135296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.135296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.135334 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.135533 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.135534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.135534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.135647 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.135701 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.136095 (+ 394us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.136146 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195}
I20260926 14:56:20.136245 13966 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8f21b88955a146c9a566f880dafac879 (127.7.37.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: NON_VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: true } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: NON_VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: true } } }
I20260926 14:56:20.136444 14150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.107360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:33917 (local address 127.0.0.1:41408)
0926 14:56:20.107635 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.107643 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.107688 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.107899 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.107900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.108045 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:20.108093 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.108094 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.108939 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.108939 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.109710 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.109711 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.110158 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.110159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.110238 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.132159 (+ 21921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:20.132164 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:20.132571 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:20.134191 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.134191 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.134192 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.134375 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.134582 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.134583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.134583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.134843 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.135905 (+ 1062us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:20.135906 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:20.135939 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.136279 (+ 340us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.136361 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":3,"thread_start_us":77,"threads_started":1}
I20260926 14:56:20.137073 14100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:20.137182 13966 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.37.252:39947
I20260926 14:56:20.137248 13966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0e48c6b89a3847f2835be52b19a380ff" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: NON_VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: true } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: NON_VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: true } } }
I20260926 14:56:20.137305 13966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:20.137511 14058 master.cc:537] Master@127.7.37.252:39947 shutting down...
I20260926 14:56:20.137882 14100 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 1 FOLLOWER]: Refusing update from remote peer 0e48c6b89a3847f2835be52b19a380ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 14:56:20.138605 13965 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: NON_VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 14:56:20.139164 13390 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff -> Peer 8f21b88955a146c9a566f880dafac879 (127.7.37.252:39947): Couldn't send request to peer 8f21b88955a146c9a566f880dafac879. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 14:56:20.139890 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.136015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:33917 (local address 127.0.0.1:41412)
0926 14:56:20.136797 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.136807 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.136853 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.137013 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.137013 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.137015 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:20.137046 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.137047 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.137829 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.137831 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.138963 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.138966 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.139645 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.139646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.139701 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.139721 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.139797 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.139831 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768}
I20260926 14:56:20.140228 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.136153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41412 (local address 127.7.37.253:33917)
0926 14:56:20.136264 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.136265 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.136843 (+ 578us) server_negotiation.cc:407] Connection header received
0926 14:56:20.136896 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.136897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.136917 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.136965 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:20.137110 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.137740 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.139045 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.139573 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.139638 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.139712 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.140138 (+ 426us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.140170 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
I20260926 14:56:20.140537 14014 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 1 LEARNER]: Refusing update from remote peer 0e48c6b89a3847f2835be52b19a380ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 14:56:20.140816 13965 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: NON_VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 14:56:20.141433 14014 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8f21b88955a146c9a566f880dafac879 (127.7.37.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: NON_VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: true } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: NON_VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: true } } }
I20260926 14:56:20.142427 14154 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e48c6b89a3847f2835be52b19a380ff. Latest consensus state: current_term: 1 leader_uuid: "0e48c6b89a3847f2835be52b19a380ff" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: NON_VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: true } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: NON_VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: true } } }
I20260926 14:56:20.142526 14058 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:20.142554 14154 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [sys.catalog]: This master's current role is: LEARNER
I20260926 14:56:20.142609 14058 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:20.142657 14058 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: stopping tablet replica
I20260926 14:56:20.142439 14043 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0e48c6b89a3847f2835be52b19a380ff" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: NON_VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: true } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: NON_VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: true } } }
I20260926 14:56:20.142880 14043 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [sys.catalog]: This master's current role is: LEARNER
I20260926 14:56:20.142989 13965 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff: attempting to promote NON_VOTER e3a46c25f46f453d9991a98b0f3ecff8 to VOTER
I20260926 14:56:20.143139 13965 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: NON_VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: true } }
I20260926 14:56:20.143544 14014 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 1 LEARNER]: Refusing update from remote peer 0e48c6b89a3847f2835be52b19a380ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260926 14:56:20.143823 13965 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260926 14:56:20.144307 13359 connection.cc:535] client connection to 127.7.37.252:39947 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 14:56:20.144431 13390 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff -> Peer 8f21b88955a146c9a566f880dafac879 (127.7.37.252:39947): Couldn't send request to peer 8f21b88955a146c9a566f880dafac879. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 14:56:20.145033 13966 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e3a46c25f46f453d9991a98b0f3ecff8 (127.7.37.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: NON_VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: true } } }
I20260926 14:56:20.145169 14014 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e3a46c25f46f453d9991a98b0f3ecff8 (127.7.37.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: NON_VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: true } } }
I20260926 14:56:20.145782 13966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0e48c6b89a3847f2835be52b19a380ff" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: NON_VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: true } } }
I20260926 14:56:20.145853 13966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:20.145905 13966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0e48c6b89a3847f2835be52b19a380ff" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: NON_VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: true } } }
I20260926 14:56:20.145951 13966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:20.146018 14043 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0e48c6b89a3847f2835be52b19a380ff" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: NON_VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: true } } }
I20260926 14:56:20.146102 14043 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 14:56:20.155652 14058 master.cc:552] Master@127.7.37.252:39947 shutdown complete.
I20260926 14:56:20.155694 14058 master_runner.cc:305] Clearing existing system tablet
I20260926 14:56:20.155855 14058 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:20.155898 14058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:20.156493 14058 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:20.156929 14158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:20.157037 14058 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:20.157080 14058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:20.157102 14058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/wal
uuid: "8f21b88955a146c9a566f880dafac879"
format_stamp: "Formatted at 2026-09-26 14:56:20 on dist-test-slave-f9xb"
server_key: "8b5dc78044bb4a86e0465b7a2535441b"
server_key_iv: "daa355e115c88b371be2f74d9d908720"
server_key_version: "encryptionkey@0"
I20260926 14:56:20.157615 14058 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:20.158661 14058 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 14:56:20.158715 14058 log.cc:1196] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260926 14:56:20.158871 14058 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: Deleting consensus metadata
I20260926 14:56:20.158967 14058 master_runner.cc:315] Copying system tablet from 127.7.37.254:34241
I20260926 14:56:20.159430 14058 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: tablet copy: Beginning tablet copy session from remote peer at address 127.7.37.254:34241
I20260926 14:56:20.165257 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.159710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52316 (local address 127.7.37.254:34241)
0926 14:56:20.159764 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.159765 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.159803 (+ 38us) server_negotiation.cc:407] Connection header received
0926 14:56:20.159859 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.159859 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.159867 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.159908 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.160182 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.160690 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.161657 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.161824 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.161869 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.163059 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.163060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.163060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.163066 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.164317 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.164696 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.164697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.164697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.164747 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.164907 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.164907 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.164908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.164992 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.165035 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.165156 (+ 121us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.165205 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 14:56:20.165326 14164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.159643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:52316)
0926 14:56:20.159731 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.159760 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.159815 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.159912 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.159913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.160073 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:20.160117 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.160118 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.160754 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.160755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.161580 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.161582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.161880 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.161881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.161938 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.162649 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:20.162653 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:20.162971 (+ 318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:20.164428 (+ 1457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.164429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.164429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.164618 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.164798 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.164799 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.164799 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.164858 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.165038 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:20.165039 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:20.165078 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.165140 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.165227 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"thread_start_us":51,"threads_started":1}
I20260926 14:56:20.165557 13426 tablet_copy_service.cc:140] P 0e48c6b89a3847f2835be52b19a380ff: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8f21b88955a146c9a566f880dafac879 ({username='oryx', principal='oryx/127.7.37.252@KRBTEST.COM'} at 127.0.0.1:52316)
I20260926 14:56:20.165601 13426 tablet_copy_service.cc:161] P 0e48c6b89a3847f2835be52b19a380ff: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8f21b88955a146c9a566f880dafac879 at {username='oryx', principal='oryx/127.7.37.252@KRBTEST.COM'} at 127.0.0.1:52316: session id = 8f21b88955a146c9a566f880dafac879-00000000000000000000000000000000
I20260926 14:56:20.165952 13426 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 14:56:20.166320 14058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 14:56:20.167099 14058 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: tablet copy: Starting download of 0 data blocks...
I20260926 14:56:20.167225 14058 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: tablet copy: Starting download of 1 WAL segments...
I20260926 14:56:20.167910 14058 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 14:56:20.168581 13426 tablet_copy_service.cc:342] P 0e48c6b89a3847f2835be52b19a380ff: Request end of tablet copy session 8f21b88955a146c9a566f880dafac879-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.37.252@KRBTEST.COM'} at 127.0.0.1:52316
I20260926 14:56:20.168622 13426 tablet_copy_service.cc:433] P 0e48c6b89a3847f2835be52b19a380ff: ending tablet copy session 8f21b88955a146c9a566f880dafac879-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8f21b88955a146c9a566f880dafac879
I20260926 14:56:20.170727 14058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:20.171774 14165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:20.171857 14168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:20.171873 14166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:20.172076 14058 server_base.cc:1034] running on GCE node
I20260926 14:56:20.172166 14058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:20.172278 14058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:20.173424 14058 hybrid_clock.cc:648] HybridClock initialized: now 1790434580173415 us; error 20 us; skew 500 ppm
Sep 26 14:56:20 dist-test-slave-f9xb krb5kdc[13361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434580, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:20.174521 14058 init.cc:377] Logged in from keytab as oryx/127.7.37.252@KRBTEST.COM (short username oryx)
I20260926 14:56:20.174733 14058 webserver.cc:460] Webserver started at http://127.7.37.252:43581/ using document root <none> and password file <none>
I20260926 14:56:20.174841 14058 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:20.174870 14058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:20.175351 14058 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:20.175766 14175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:20.175884 14058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:20.175925 14058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/wal
uuid: "8f21b88955a146c9a566f880dafac879"
format_stamp: "Formatted at 2026-09-26 14:56:20 on dist-test-slave-f9xb"
server_key: "8b5dc78044bb4a86e0465b7a2535441b"
server_key_iv: "daa355e115c88b371be2f74d9d908720"
server_key_version: "encryptionkey@0"
I20260926 14:56:20.175966 14058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:20.190996 14058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:20.192369 14058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:20.194933 14058 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.252:39947
I20260926 14:56:20.195086 14227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.252:39947 every 8 connection(s)
I20260926 14:56:20.195204 14058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/master-2/data/info.pb
I20260926 14:56:20.195813 14228 sys_catalog.cc:263] Verifying existing consensus state
I20260926 14:56:20.196206 14228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: Bootstrap starting.
I20260926 14:56:20.198319 14228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 14:56:20.198496 14228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: Bootstrap complete.
I20260926 14:56:20.198639 14228 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: NON_VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: true } }
I20260926 14:56:20.198707 14228 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8f21b88955a146c9a566f880dafac879, State: Initialized, Role: LEARNER
I20260926 14:56:20.198731 14228 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: NON_VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: true } }
I20260926 14:56:20.198881 14228 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:20.198886 14231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: NON_VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: true } } }
I20260926 14:56:20.199026 14231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [sys.catalog]: This master's current role is: LEARNER
I20260926 14:56:20.200107 14242 catalog_manager.cc:1259] Loaded cluster ID: dcfe92ec089b4fe7817234b268b008d2
I20260926 14:56:20.200151 14242 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: loading cluster ID for follower catalog manager: success
I20260926 14:56:20.201210 14242 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: acquiring CA information for follower catalog manager: success
I20260926 14:56:20.201499 14242 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 14:56:20.250483 14243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.245231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54270 (local address 127.7.37.252:39947)
0926 14:56:20.245339 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.245342 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.245349 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:20.245369 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.245370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.245377 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.245430 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.245865 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.246451 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.247134 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.247322 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.247388 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.248515 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.248517 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.248517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.248523 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.249647 (+ 1124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.249933 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.249934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.249934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.249981 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.250107 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.250108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.250108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.250194 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.250263 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.250332 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.250395 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":74,"threads_started":1}
I20260926 14:56:20.250957 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 13468
I20260926 14:56:20.254731 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:42405
--local_ip_for_outbound_sockets=127.7.37.193
--tserver_master_addrs=127.7.37.254:34241
--webserver_port=42999
--webserver_interface=127.7.37.193
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.193.keytab
--principal=oryx/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.37.254:34241,127.7.37.253:33917
--tserver_master_addrs=127.7.37.254:34241,127.7.37.253:33917,127.7.37.252:39947 with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:20.330837 14244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:20.331002 14244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:20.331035 14244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:20.332122 14244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:20.332154 14244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:20.332176 14244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:56:20.333525 14244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.193.keytab
--principal=oryx/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:42405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=42999
--tserver_master_addrs=127.7.37.254:34241,127.7.37.253:33917,127.7.37.252:39947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:20.333683 14244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:20.333837 14244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:20.338124 14249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:20.338225 14244 server_base.cc:1034] running on GCE node
W20260926 14:56:20.338114 14252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:20.338105 14250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:20.338568 14244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:20.338759 14244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:20.339888 14244 hybrid_clock.cc:648] HybridClock initialized: now 1790434580339880 us; error 36 us; skew 500 ppm
Sep 26 14:56:20 dist-test-slave-f9xb krb5kdc[13361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434580, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:20.342299 14244 init.cc:377] Logged in from keytab as oryx/127.7.37.193@KRBTEST.COM (short username oryx)
I20260926 14:56:20.342751 14244 webserver.cc:460] Webserver started at http://127.7.37.193:42999/ using document root <none> and password file <none>
I20260926 14:56:20.342975 14244 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:20.343047 14244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:20.344645 14244 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:20.345278 14259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:20.345422 14244 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:20.345499 14244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "8ae81595f9464cadabc6fc7748af9c70"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "1e45f2b6ec6f64c90f0d109e43289a42"
server_key_iv: "f4f7c04d519a5f18990da2e2bc2b8a0e"
server_key_version: "encryptionkey@0"
I20260926 14:56:20.345767 14244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:20.356956 14244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:20.358485 14244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:20.358590 14244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:20.358752 14244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:20.358987 14244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:20.359016 14244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:20.359036 14244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:20.359058 14244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:20.364085 14244 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:42405
I20260926 14:56:20.364116 14372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:42405 every 8 connection(s)
I20260926 14:56:20.364413 14244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:56:20.368417 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 14244
I20260926 14:56:20.368548 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 13602
Sep 26 14:56:20 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434580, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM
Sep 26 14:56:20 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434580, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.253@KRBTEST.COM
Sep 26 14:56:20 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434580, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.252@KRBTEST.COM
I20260926 14:56:20.374814 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:42005
--local_ip_for_outbound_sockets=127.7.37.194
--tserver_master_addrs=127.7.37.254:34241
--webserver_port=44645
--webserver_interface=127.7.37.194
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.194.keytab
--principal=oryx/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.37.254:34241,127.7.37.253:33917
--tserver_master_addrs=127.7.37.254:34241,127.7.37.253:33917,127.7.37.252:39947 with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 14:56:20.375083 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.364951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:58549 (local address 127.7.37.254:34241)
0926 14:56:20.364995 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.364997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.365324 (+ 327us) server_negotiation.cc:407] Connection header received
0926 14:56:20.366151 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.366152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.366164 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.366214 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.366873 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.367636 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.368669 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.368962 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.369000 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.372105 (+ 3105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.372107 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.372108 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.372114 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.373650 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.374147 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.374148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.374149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.374201 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.374450 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.374450 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.374451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.374566 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.374605 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.374938 (+ 333us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.375006 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:56:20.375507 14378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.364952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.193:58549)
0926 14:56:20.365262 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.365271 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.366063 (+ 792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.366281 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.366285 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.366609 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:20.366796 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.366800 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.367712 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.367713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.368573 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.368575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.369079 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.369079 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.369233 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.369718 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:20.369731 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:20.372013 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:20.373809 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.373815 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.373817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.374068 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.374263 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.374264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.374265 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.374384 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.374648 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:20.374654 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:20.374790 (+ 136us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.375163 (+ 373us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.375325 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":2,"thread_start_us":211,"threads_started":1}
I20260926 14:56:20.375566 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.365478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:58833 (local address 127.7.37.253:33917)
0926 14:56:20.365526 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.365528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.365536 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:20.366173 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.366174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.366186 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.366216 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.366902 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.367370 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.368668 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.368966 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.369000 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.372097 (+ 3097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.372099 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.372099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.372106 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.374389 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.374821 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.374822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.374822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.374883 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.375069 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.375070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.375070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.375196 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.375233 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.375415 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.375484 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 14:56:20.375885 14379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.365319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:33917 (local address 127.7.37.193:58833)
0926 14:56:20.365414 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.365421 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.366077 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.366284 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.366285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.366637 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:20.366798 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.366804 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.367458 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.367458 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.368573 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.368575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.369080 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.369081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.369233 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.370079 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:20.370084 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:20.371997 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:20.374477 (+ 2480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.374477 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.374478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.374746 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.374949 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.374950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.374950 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.375002 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.375272 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:20.375273 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:20.375310 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.375636 (+ 326us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.375783 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":13,"thread_start_us":59,"threads_started":1}
I20260926 14:56:20.376222 14375 heartbeater.cc:347] Connected to a master server at 127.7.37.253:33917
I20260926 14:56:20.376305 14375 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:20.376462 14375 heartbeater.cc:511] Master 127.7.37.253:33917 requested a full tablet report, sending...
I20260926 14:56:20.376575 14377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.364951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:39947 (local address 127.7.37.193:60773)
0926 14:56:20.365198 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.365233 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.366061 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.366281 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.366286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.366636 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:20.366798 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.366804 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.369456 (+ 2652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.369458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.370807 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.370809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.371205 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.371206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.371262 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.371606 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:20.371610 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:20.373005 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:20.374563 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.374563 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.374564 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.374798 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.375343 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.375343 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.375344 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.375408 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.376241 (+ 833us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:20.376242 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:20.376325 (+ 83us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.376394 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.376474 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":22,"thread_start_us":193,"threads_started":1}
I20260926 14:56:20.376581 14243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.365120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:60773 (local address 127.7.37.252:39947)
0926 14:56:20.365164 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.365166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.365315 (+ 149us) server_negotiation.cc:407] Connection header received
0926 14:56:20.366173 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.366174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.366186 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.366216 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.366900 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.367635 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.370883 (+ 3248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.371144 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.371198 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.373098 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.373099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.373099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.373106 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.374493 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.374870 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.374871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.374872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.374918 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.375474 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.375475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.375476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.375585 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.375622 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.376430 (+ 808us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.376498 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:56:20.376951 14004 ts_manager.cc:194] Registered new tserver with Master: 8ae81595f9464cadabc6fc7748af9c70 (127.7.37.193:42405)
I20260926 14:56:20.377027 14373 heartbeater.cc:347] Connected to a master server at 127.7.37.252:39947
I20260926 14:56:20.377077 14373 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:20.377131 14373 heartbeater.cc:511] Master 127.7.37.252:39947 requested a full tablet report, sending...
I20260926 14:56:20.377188 14374 heartbeater.cc:347] Connected to a master server at 127.7.37.254:34241
I20260926 14:56:20.377233 14374 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:20.377292 14374 heartbeater.cc:511] Master 127.7.37.254:34241 requested a full tablet report, sending...
I20260926 14:56:20.377434 14192 ts_manager.cc:194] Registered new tserver with Master: 8ae81595f9464cadabc6fc7748af9c70 (127.7.37.193:42405)
I20260926 14:56:20.377552 13406 ts_manager.cc:194] Re-registered known tserver with Master: 8ae81595f9464cadabc6fc7748af9c70 (127.7.37.193:42405)
I20260926 14:56:20.378000 13406 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:58549
W20260926 14:56:20.451576 14380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:20.451735 14380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:20.451793 14380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:20.452929 14380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:20.452967 14380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:20.452991 14380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:56:20.454326 14380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.194.keytab
--principal=oryx/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:42005
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=44645
--tserver_master_addrs=127.7.37.254:34241,127.7.37.253:33917,127.7.37.252:39947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:20.454483 14380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:20.454633 14380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:20.458870 14386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:20.458956 14388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:20.458981 14380 server_base.cc:1034] running on GCE node
W20260926 14:56:20.458896 14385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:20.459213 14380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:20.459462 14380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:20.460618 14380 hybrid_clock.cc:648] HybridClock initialized: now 1790434580460593 us; error 33 us; skew 500 ppm
Sep 26 14:56:20 dist-test-slave-f9xb krb5kdc[13361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434580, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:20.469852 14380 init.cc:377] Logged in from keytab as oryx/127.7.37.194@KRBTEST.COM (short username oryx)
I20260926 14:56:20.470199 14380 webserver.cc:460] Webserver started at http://127.7.37.194:44645/ using document root <none> and password file <none>
I20260926 14:56:20.470391 14380 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:20.470449 14380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:20.471587 14380 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 14:56:20.472086 14395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:20.472213 14380 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:20.472293 14380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "1445225627d3420ea69437d8763866b7"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "687f98c735481d4485c434d3060e7f1e"
server_key_iv: "b974bd24be4ba9842f8d36b9b78583d8"
server_key_version: "encryptionkey@0"
I20260926 14:56:20.472546 14380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:20.481453 14380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:20.483258 14380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:20.483373 14380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:20.483536 14380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:20.483804 14380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:20.483841 14380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:20.483866 14380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:20.483889 14380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:20.489091 14380 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:42005
I20260926 14:56:20.489143 14508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:42005 every 8 connection(s)
I20260926 14:56:20.489394 14380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:56:20.489779 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 14380
I20260926 14:56:20.489923 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 13736
I20260926 14:56:20.495214 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:42945
--local_ip_for_outbound_sockets=127.7.37.195
--tserver_master_addrs=127.7.37.254:34241
--webserver_port=38979
--webserver_interface=127.7.37.195
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.195.keytab
--principal=oryx/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.37.254:34241,127.7.37.253:33917
--tserver_master_addrs=127.7.37.254:34241,127.7.37.253:33917,127.7.37.252:39947 with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 14:56:20 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434580, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.252@KRBTEST.COM
Sep 26 14:56:20 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434580, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM
Sep 26 14:56:20 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434580, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.253@KRBTEST.COM
I20260926 14:56:20.501295 14514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.489986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.194:58439)
0926 14:56:20.490350 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.490359 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.491372 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.491627 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.491631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.492087 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:20.492367 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.492372 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.493340 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.493341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.494632 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.494635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.495135 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.495137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.495269 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.495817 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:20.495826 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:20.497893 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:20.499689 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.499695 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.499697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.500073 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.500291 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.500292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.500292 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.500408 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.500687 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:20.500693 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:20.500836 (+ 143us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.500929 (+ 93us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.501077 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":122,"thread_start_us":70,"threads_started":1}
I20260926 14:56:20.501312 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.491139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:39771 (local address 127.7.37.253:33917)
0926 14:56:20.491180 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.491182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.492061 (+ 879us) server_negotiation.cc:407] Connection header received
0926 14:56:20.492153 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.492154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.492164 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.492189 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.492508 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.493218 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.494724 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.495017 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.495052 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.498861 (+ 3809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.498862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.498863 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.498870 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.500317 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.500683 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.500684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.500685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.500733 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.500888 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.500888 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.500889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.500985 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.501035 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.501159 (+ 124us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.501224 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 14:56:20.501369 14515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.490978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:33917 (local address 127.7.37.194:39771)
0926 14:56:20.491997 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.492006 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.492087 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.492242 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.492243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.492382 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:20.492433 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.492434 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.493322 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.493323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.494631 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.494633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.495131 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.495132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.495271 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.495882 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:20.495887 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:20.498769 (+ 2882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:20.500420 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.500421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.500422 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.500606 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.500786 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.500786 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.500787 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.500839 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.501034 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:20.501035 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:20.501073 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.501133 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.501225 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1005,"mutex_wait_us":169,"thread_start_us":77,"threads_started":1}
I20260926 14:56:20.501678 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.490166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:58439 (local address 127.7.37.254:34241)
0926 14:56:20.490205 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.490207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.490417 (+ 210us) server_negotiation.cc:407] Connection header received
0926 14:56:20.491517 (+ 1100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.491519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.491532 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.491582 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.492476 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.493240 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.494724 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.495008 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.495073 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.497973 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.497974 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.497974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.497981 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.499588 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.500154 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.500155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.500155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.500214 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.500480 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.500481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.500481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.500607 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.500662 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.501521 (+ 859us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.501595 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:56:20.502151 14510 heartbeater.cc:347] Connected to a master server at 127.7.37.254:34241
I20260926 14:56:20.502219 14510 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:20.502354 14510 heartbeater.cc:511] Master 127.7.37.254:34241 requested a full tablet report, sending...
I20260926 14:56:20.502630 13406 ts_manager.cc:194] Re-registered known tserver with Master: 1445225627d3420ea69437d8763866b7 (127.7.37.194:42005)
I20260926 14:56:20.502914 14243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.490206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:60147 (local address 127.7.37.252:39947)
0926 14:56:20.490248 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.490250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.490383 (+ 133us) server_negotiation.cc:407] Connection header received
0926 14:56:20.491467 (+ 1084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.491468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.491480 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.491506 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.492459 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.493201 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.494723 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.495012 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.495052 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.498445 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.498446 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.498447 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.498453 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.500553 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.502148 (+ 1595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.502149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.502149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.502204 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.502411 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.502412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.502412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.502528 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.502569 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.502756 (+ 187us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.502828 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:56:20.502998 13406 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:58439
I20260926 14:56:20.503140 14511 heartbeater.cc:347] Connected to a master server at 127.7.37.253:33917
I20260926 14:56:20.503190 14511 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:20.503211 14513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.489986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:39947 (local address 127.7.37.194:60147)
0926 14:56:20.490285 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.490316 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.491358 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.491625 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.491629 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.492084 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:20.492367 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.492372 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.493322 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.493323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.494630 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.494633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.495130 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.495131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.495271 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.496000 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:20.496004 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:20.498379 (+ 2375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:20.501863 (+ 3484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.501871 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.501872 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.502060 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.502286 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.502287 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.502288 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.502342 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.502613 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:20.502614 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:20.502651 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.502993 (+ 342us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.503078 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":131,"thread_start_us":84,"threads_started":1}
I20260926 14:56:20.503240 14511 heartbeater.cc:511] Master 127.7.37.253:33917 requested a full tablet report, sending...
I20260926 14:56:20.503536 14509 heartbeater.cc:347] Connected to a master server at 127.7.37.252:39947
I20260926 14:56:20.503556 14004 ts_manager.cc:194] Registered new tserver with Master: 1445225627d3420ea69437d8763866b7 (127.7.37.194:42005)
I20260926 14:56:20.503572 14509 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:20.503621 14509 heartbeater.cc:511] Master 127.7.37.252:39947 requested a full tablet report, sending...
I20260926 14:56:20.504040 14192 ts_manager.cc:194] Registered new tserver with Master: 1445225627d3420ea69437d8763866b7 (127.7.37.194:42005)
W20260926 14:56:20.572850 14516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:20.573016 14516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:20.573037 14516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:20.574117 14516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:20.574155 14516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:20.574182 14516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:56:20.575634 14516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.195.keytab
--principal=oryx/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:42945
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=38979
--tserver_master_addrs=127.7.37.254:34241,127.7.37.253:33917,127.7.37.252:39947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:20.575877 14516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:20.576066 14516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:20.580438 14521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:20.580497 14522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:20.580588 14516 server_base.cc:1034] running on GCE node
W20260926 14:56:20.580435 14524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:20.580797 14516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:20.581012 14516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:20.582140 14516 hybrid_clock.cc:648] HybridClock initialized: now 1790434580582127 us; error 28 us; skew 500 ppm
Sep 26 14:56:20 dist-test-slave-f9xb krb5kdc[13361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434580, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:20.584836 14516 init.cc:377] Logged in from keytab as oryx/127.7.37.195@KRBTEST.COM (short username oryx)
I20260926 14:56:20.585268 14516 webserver.cc:460] Webserver started at http://127.7.37.195:38979/ using document root <none> and password file <none>
I20260926 14:56:20.585490 14516 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:20.585558 14516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:20.585875 14243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.583007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54274 (local address 127.7.37.252:39947)
0926 14:56:20.583055 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.583057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.583064 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:20.583088 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.583089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.583110 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.583169 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:20.583290 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.584056 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.584984 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.585564 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.585633 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.585704 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.585769 (+ 65us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.585811 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 14:56:20.585873 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.582925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:39947 (local address 127.0.0.1:54274)
0926 14:56:20.582969 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.582978 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.583022 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.583187 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.583189 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.583190 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:20.583231 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.583232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.584124 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.584124 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.584866 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.584869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.585637 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.585638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.585710 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.585714 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.585766 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.585813 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260926 14:56:20.586181 14202 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260926 14:56:20.586301 14231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e48c6b89a3847f2835be52b19a380ff. Latest consensus state: current_term: 1 leader_uuid: "0e48c6b89a3847f2835be52b19a380ff" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: NON_VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: true } } }
I20260926 14:56:20.586416 14231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [sys.catalog]: This master's current role is: LEARNER
I20260926 14:56:20.587005 14516 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:20.587469 14531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:20.587602 14516 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:20.587682 14516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "356c4e7736584de99be4f7f0a33b2db1"
format_stamp: "Formatted at 2026-09-26 14:56:19 on dist-test-slave-f9xb"
server_key: "540a4d0ce9ed1d709e4e0ad5add96c8e"
server_key_iv: "9bbb6ba93dea14000c1cb530832490cc"
server_key_version: "encryptionkey@0"
I20260926 14:56:20.587975 14516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:20.606379 14516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:20.607880 14516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:20.607997 14516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:20.608170 14516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:20.608412 14516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:20.608441 14516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:20.608466 14516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:20.608487 14516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:20.613741 14516 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:42945
I20260926 14:56:20.613773 14644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:42945 every 8 connection(s)
I20260926 14:56:20.614045 14516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:56:20.620059 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 14516
Sep 26 14:56:20 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434580, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.253@KRBTEST.COM
Sep 26 14:56:20 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434580, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.252@KRBTEST.COM
Sep 26 14:56:20 dist-test-slave-f9xb krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434580, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM
I20260926 14:56:20.625185 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.614645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:40975 (local address 127.7.37.254:34241)
0926 14:56:20.614700 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.614702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.615072 (+ 370us) server_negotiation.cc:407] Connection header received
0926 14:56:20.615863 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.615864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.615874 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.615903 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.616533 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.617248 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.619201 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.619503 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.619572 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.621969 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.621971 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.621971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.621977 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.623652 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.624235 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.624236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.624236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.624288 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.624543 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.624544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.624544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.624665 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.624715 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.625028 (+ 313us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.625097 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 14:56:20.625278 14649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.614893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:39947 (local address 127.7.37.195:56155)
0926 14:56:20.615134 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.615143 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.615760 (+ 617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.615976 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.615976 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.616249 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:20.616427 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.616433 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.617845 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.617846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.618601 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.618603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.619091 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.619092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.619147 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.619500 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:20.619504 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:20.622254 (+ 2750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:20.623901 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.623902 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.623902 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.624115 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.624379 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.624380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.624380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.624475 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.624707 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:20.624711 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:20.624848 (+ 137us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.624940 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.625085 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":2,"thread_start_us":60,"threads_started":1}
I20260926 14:56:20.625550 14243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.615024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:56155 (local address 127.7.37.252:39947)
0926 14:56:20.615058 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.615059 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.615188 (+ 129us) server_negotiation.cc:407] Connection header received
0926 14:56:20.615862 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.615863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.615874 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.615903 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.617328 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.617772 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.618684 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.618990 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.619025 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.622332 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.622333 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.622334 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.622340 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.623826 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.624254 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.624255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.624255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.624319 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.624532 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.624533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.624533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.624641 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.624688 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.625400 (+ 712us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.625467 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260926 14:56:20.625756 14651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.614636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.195:40975)
0926 14:56:20.615007 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.615015 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.615760 (+ 745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.615963 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.615964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.616248 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:20.616449 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.616450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.618385 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.618385 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.619131 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.619132 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.619579 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.619580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.619643 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.620052 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:20.620057 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:20.621869 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:20.623728 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.623729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.623730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.624189 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.624357 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.624358 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.624358 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.624474 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.624736 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:20.624737 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:20.624847 (+ 110us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.624940 (+ 93us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.625658 (+ 718us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":624,"thread_start_us":79,"threads_started":1}
I20260926 14:56:20.625808 14645 heartbeater.cc:347] Connected to a master server at 127.7.37.252:39947
I20260926 14:56:20.625877 14645 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:20.626014 14645 heartbeater.cc:511] Master 127.7.37.252:39947 requested a full tablet report, sending...
I20260926 14:56:20.626061 14646 heartbeater.cc:347] Connected to a master server at 127.7.37.254:34241
I20260926 14:56:20.626103 14646 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:20.626159 14646 heartbeater.cc:511] Master 127.7.37.254:34241 requested a full tablet report, sending...
I20260926 14:56:20.626345 13406 ts_manager.cc:194] Re-registered known tserver with Master: 356c4e7736584de99be4f7f0a33b2db1 (127.7.37.195:42945)
I20260926 14:56:20.626368 14192 ts_manager.cc:194] Registered new tserver with Master: 356c4e7736584de99be4f7f0a33b2db1 (127.7.37.195:42945)
I20260926 14:56:20.626778 13406 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:40975
I20260926 14:56:20.627271 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.614700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:34599 (local address 127.7.37.253:33917)
0926 14:56:20.614738 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.614740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.614884 (+ 144us) server_negotiation.cc:407] Connection header received
0926 14:56:20.615829 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.615830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.615848 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.615895 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.616506 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.617239 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.618380 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.618561 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.618633 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.622111 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.622112 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.622113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.622121 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.623516 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.626546 (+ 3030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.626547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.626548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.626602 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.626786 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.626787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.626787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.626897 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.626944 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.627116 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.627181 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 14:56:20.627553 14650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.614635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:33917 (local address 127.7.37.195:34599)
0926 14:56:20.614772 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.614806 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.615725 (+ 919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.615954 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.615957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.616238 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:20.616425 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.616429 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.617324 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.617324 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.618295 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.618297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.618650 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.618651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.618767 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.619165 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:20.619174 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:20.622015 (+ 2841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:20.623596 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.623601 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.623603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.626469 (+ 2866us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.626664 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:20.626665 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:20.626665 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:20.626718 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:20.626974 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:20.626975 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:20.627012 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.627380 (+ 368us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.627465 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":2,"thread_start_us":85,"threads_started":1}
I20260926 14:56:20.627964 14647 heartbeater.cc:347] Connected to a master server at 127.7.37.253:33917
I20260926 14:56:20.628006 14647 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:20.628062 14647 heartbeater.cc:511] Master 127.7.37.253:33917 requested a full tablet report, sending...
I20260926 14:56:20.628309 14004 ts_manager.cc:194] Registered new tserver with Master: 356c4e7736584de99be4f7f0a33b2db1 (127.7.37.195:42945)
I20260926 14:56:20.630091 14152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.621518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41420 (local address 127.7.37.253:33917)
0926 14:56:20.621553 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.621555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.621563 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:20.621581 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.621581 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.621591 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.621648 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.623373 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.624095 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.625040 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.625323 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.625382 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.627326 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.627328 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.627328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.627334 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.628933 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.629264 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.629265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.629265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.629316 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.629490 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.629491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.629491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.629603 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.629687 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.629932 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.630008 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 14:56:20.630708 14657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.621103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52324 (local address 127.7.37.254:34241)
0926 14:56:20.621230 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.621232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.621589 (+ 357us) server_negotiation.cc:407] Connection header received
0926 14:56:20.621676 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.621677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.621688 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.621727 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.624012 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.624717 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.625805 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.626078 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.626116 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.627479 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.627480 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.627481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.627487 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.629173 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.629451 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.629452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.629452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.629504 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.629649 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.629649 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.629649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.629755 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.629849 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.630588 (+ 739us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.630653 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260926 14:56:20.632246 13406 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:56:20.633627 13406 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:56:20.634159 14660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.621288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54280 (local address 127.7.37.252:39947)
0926 14:56:20.621406 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.621407 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.622320 (+ 913us) server_negotiation.cc:407] Connection header received
0926 14:56:20.622341 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.622342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.622354 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.622404 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.628368 (+ 5964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.629642 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.630498 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.630803 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.630857 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.632075 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.632076 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.632077 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.632083 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.633265 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.633542 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.633543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.633543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.633593 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.633744 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.633745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.633745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.633866 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.633963 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.634028 (+ 65us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.634096 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":42,"threads_started":1}
I20260926 14:56:20.634294 14156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0e48c6b89a3847f2835be52b19a380ff" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: NON_VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: true } } }
I20260926 14:56:20.634388 14156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:20.635273 14155 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff: attempting to promote NON_VOTER 8f21b88955a146c9a566f880dafac879 to VOTER
I20260926 14:56:20.635394 14155 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } }
I20260926 14:56:20.635735 14014 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 1 FOLLOWER]: Refusing update from remote peer 0e48c6b89a3847f2835be52b19a380ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260926 14:56:20.635790 14202 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 1 LEARNER]: Refusing update from remote peer 0e48c6b89a3847f2835be52b19a380ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260926 14:56:20.635931 14155 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260926 14:56:20.636034 13965 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260926 14:56:20.636557 14155 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 8f21b88955a146c9a566f880dafac879 (127.7.37.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } } }
I20260926 14:56:20.636751 14014 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 8f21b88955a146c9a566f880dafac879 (127.7.37.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } } }
I20260926 14:56:20.637444 14155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0e48c6b89a3847f2835be52b19a380ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } } }
I20260926 14:56:20.637480 14043 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0e48c6b89a3847f2835be52b19a380ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } } }
I20260926 14:56:20.637519 14155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:20.637537 14043 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 14:56:20.637570 14155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0e48c6b89a3847f2835be52b19a380ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } } }
I20260926 14:56:20.637611 14155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:20.637589 13966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0e48c6b89a3847f2835be52b19a380ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } } }
I20260926 14:56:20.637655 13966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:20.637758 14202 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 8f21b88955a146c9a566f880dafac879 (127.7.37.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } } }
I20260926 14:56:20.638655 14231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0e48c6b89a3847f2835be52b19a380ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } } }
I20260926 14:56:20.638734 14231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 14:56:20.642920 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.639803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:42945 (local address 127.0.0.1:56824)
0926 14:56:20.639834 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.639843 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.639877 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.640259 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.640260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.640261 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:20.640301 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.640302 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.641062 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.641063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.641890 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.641892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.642715 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.642716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.642771 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.642774 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.642826 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.642859 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260926 14:56:20.642940 14671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.640011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56824 (local address 127.7.37.195:42945)
0926 14:56:20.640134 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.640138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.640146 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:20.640174 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.640175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.640185 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.640253 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:20.640347 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.641001 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.641966 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.642633 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.642698 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.642765 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.642837 (+ 72us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.642867 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":72,"threads_started":1}
I20260926 14:56:20.643535 14579 tablet_service.cc:1467] Processing CreateTablet for tablet b27ad5c536dd4a17a3ed21a231f4f625 (DEFAULT_TABLE table=test-table [id=f9d632854b8d424aa33b7bb1355aee78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:20.643695 14668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.639846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:42005 (local address 127.0.0.1:44620)
0926 14:56:20.639948 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.639956 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.639989 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.640380 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.640381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.640382 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:20.640407 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.640408 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.641566 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.641567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.642656 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.642659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.643483 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.643484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.643544 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.643548 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.643599 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.643632 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":35,"thread_start_us":68,"threads_started":1}
I20260926 14:56:20.643719 14670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.639926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44620 (local address 127.7.37.194:42005)
0926 14:56:20.640219 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.640223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.640231 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:20.640266 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.640268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.640282 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.640383 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:20.640452 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.641476 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.642809 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.643402 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.643470 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.643537 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.643613 (+ 76us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.643648 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":62,"threads_started":1}
I20260926 14:56:20.643911 14579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b27ad5c536dd4a17a3ed21a231f4f625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:20.644191 14669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.639888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:42405 (local address 127.0.0.1:44796)
0926 14:56:20.640055 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.640063 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.640113 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.640474 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.640475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.640477 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:20.640515 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.640516 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.641725 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.641726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.642481 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.642483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.643615 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.643616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.643723 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.643728 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.644088 (+ 360us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.644131 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":73,"threads_started":1}
I20260926 14:56:20.644270 14443 tablet_service.cc:1467] Processing CreateTablet for tablet b27ad5c536dd4a17a3ed21a231f4f625 (DEFAULT_TABLE table=test-table [id=f9d632854b8d424aa33b7bb1355aee78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:20.644515 14672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.640152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44796 (local address 127.7.37.193:42405)
0926 14:56:20.640327 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.640331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.640338 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:20.640371 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.640373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.640386 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.640576 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:20.640630 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.641639 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.642551 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.643521 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.644299 (+ 778us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.644377 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.644402 (+ 25us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.644443 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":80,"threads_started":1}
I20260926 14:56:20.644577 14443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b27ad5c536dd4a17a3ed21a231f4f625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:20.646328 14673 tablet_bootstrap.cc:492] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1: Bootstrap starting.
I20260926 14:56:20.646862 14307 tablet_service.cc:1467] Processing CreateTablet for tablet b27ad5c536dd4a17a3ed21a231f4f625 (DEFAULT_TABLE table=test-table [id=f9d632854b8d424aa33b7bb1355aee78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:20.647122 14307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b27ad5c536dd4a17a3ed21a231f4f625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:20.647181 14673 tablet_bootstrap.cc:654] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:20.647455 14673 log.cc:824] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:20.647491 14674 tablet_bootstrap.cc:492] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7: Bootstrap starting.
I20260926 14:56:20.648150 14674 tablet_bootstrap.cc:654] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:20.648170 14673 tablet_bootstrap.cc:492] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1: No bootstrap required, opened a new log
I20260926 14:56:20.648222 14673 ts_tablet_manager.cc:1397] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:56:20.648427 14674 log.cc:824] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:20.649138 14674 tablet_bootstrap.cc:492] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7: No bootstrap required, opened a new log
I20260926 14:56:20.649204 14674 ts_tablet_manager.cc:1397] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:56:20.650049 14673 raft_consensus.cc:348] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } } peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } }
I20260926 14:56:20.650195 14673 raft_consensus.cc:374] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:20.650236 14673 raft_consensus.cc:729] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 356c4e7736584de99be4f7f0a33b2db1, State: Initialized, Role: FOLLOWER
I20260926 14:56:20.650162 14677 tablet_bootstrap.cc:492] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70: Bootstrap starting.
I20260926 14:56:20.650354 14673 consensus_queue.cc:260] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } } peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } }
I20260926 14:56:20.650601 14646 heartbeater.cc:503] Master 127.7.37.254:34241 was elected leader, sending a full tablet report...
I20260926 14:56:20.650772 14677 tablet_bootstrap.cc:654] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:20.651062 14677 log.cc:824] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:20.651345 14673 ts_tablet_manager.cc:1428] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 14:56:20.651593 14674 raft_consensus.cc:348] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } } peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } }
I20260926 14:56:20.651705 14677 tablet_bootstrap.cc:492] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70: No bootstrap required, opened a new log
I20260926 14:56:20.651731 14674 raft_consensus.cc:374] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:20.651778 14677 ts_tablet_manager.cc:1397] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:56:20.651811 14674 raft_consensus.cc:729] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1445225627d3420ea69437d8763866b7, State: Initialized, Role: FOLLOWER
I20260926 14:56:20.651930 14674 consensus_queue.cc:260] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } } peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } }
I20260926 14:56:20.652153 14674 ts_tablet_manager.cc:1428] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260926 14:56:20.652268 14510 heartbeater.cc:503] Master 127.7.37.254:34241 was elected leader, sending a full tablet report...
I20260926 14:56:20.654039 14677 raft_consensus.cc:348] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } } peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } }
I20260926 14:56:20.654147 14677 raft_consensus.cc:374] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:20.654166 14677 raft_consensus.cc:729] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ae81595f9464cadabc6fc7748af9c70, State: Initialized, Role: FOLLOWER
I20260926 14:56:20.654256 14677 consensus_queue.cc:260] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } } peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } }
I20260926 14:56:20.654462 14677 ts_tablet_manager.cc:1428] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 14:56:20.654503 14374 heartbeater.cc:503] Master 127.7.37.254:34241 was elected leader, sending a full tablet report...
W20260926 14:56:20.743942 14512 tablet.cc:2369] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:56:20.763684 14681 raft_consensus.cc:482] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:56:20.763828 14681 raft_consensus.cc:504] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } } peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } }
I20260926 14:56:20.764156 14681 leader_election.cc:288] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 356c4e7736584de99be4f7f0a33b2db1 (127.7.37.195:42945), 1445225627d3420ea69437d8763866b7 (127.7.37.194:42005)
I20260926 14:56:20.766716 14670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.764383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:54447 (local address 127.7.37.194:42005)
0926 14:56:20.764428 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.764430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.764441 (+ 11us) server_negotiation.cc:407] Connection header received
0926 14:56:20.764478 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.764479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.764489 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.764524 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:20.764677 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.765136 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.766076 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.766409 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.766458 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.766465 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.766622 (+ 157us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.766656 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 14:56:20.766726 14377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.764305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:42005 (local address 127.7.37.193:54447)
0926 14:56:20.764357 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.764368 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.764418 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.764541 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.764542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.764543 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:20.764613 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.764615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.765207 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.765207 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.765988 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.765990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.766475 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.766476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.766554 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.766558 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.766606 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.766643 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":32}
I20260926 14:56:20.766747 14671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.764419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:56195 (local address 127.7.37.195:42945)
0926 14:56:20.764461 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.764463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.764470 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:20.764494 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.764494 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.764506 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.764564 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:20.764710 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.765145 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.766079 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.766454 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.766529 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.766537 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.766655 (+ 118us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.766694 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 14:56:20.766745 14379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.764323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:42945 (local address 127.7.37.193:56195)
0926 14:56:20.764368 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:20.764378 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:20.764424 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:20.764581 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:20.764582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:20.764583 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:20.764634 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.764635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.765223 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.765223 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.765989 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:20.765991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.766534 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.766534 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:20.766585 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.766588 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:20.766637 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 14:56:20.766673 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":53}
I20260926 14:56:20.767004 14463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b27ad5c536dd4a17a3ed21a231f4f625" candidate_uuid: "8ae81595f9464cadabc6fc7748af9c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1445225627d3420ea69437d8763866b7" is_pre_election: true
I20260926 14:56:20.767032 14599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b27ad5c536dd4a17a3ed21a231f4f625" candidate_uuid: "8ae81595f9464cadabc6fc7748af9c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "356c4e7736584de99be4f7f0a33b2db1" is_pre_election: true
I20260926 14:56:20.767132 14463 raft_consensus.cc:2454] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ae81595f9464cadabc6fc7748af9c70 in term 0.
I20260926 14:56:20.767132 14599 raft_consensus.cc:2454] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ae81595f9464cadabc6fc7748af9c70 in term 0.
I20260926 14:56:20.767334 14263 leader_election.cc:302] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1445225627d3420ea69437d8763866b7, 8ae81595f9464cadabc6fc7748af9c70; no voters:
I20260926 14:56:20.767477 14681 raft_consensus.cc:2781] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:56:20.767537 14681 raft_consensus.cc:482] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:56:20.767558 14681 raft_consensus.cc:3037] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:20.768177 14681 raft_consensus.cc:504] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } } peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } }
I20260926 14:56:20.768278 14681 leader_election.cc:288] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [CANDIDATE]: Term 1 election: Requested vote from peers 356c4e7736584de99be4f7f0a33b2db1 (127.7.37.195:42945), 1445225627d3420ea69437d8763866b7 (127.7.37.194:42005)
I20260926 14:56:20.768406 14599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b27ad5c536dd4a17a3ed21a231f4f625" candidate_uuid: "8ae81595f9464cadabc6fc7748af9c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "356c4e7736584de99be4f7f0a33b2db1"
I20260926 14:56:20.768445 14463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b27ad5c536dd4a17a3ed21a231f4f625" candidate_uuid: "8ae81595f9464cadabc6fc7748af9c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1445225627d3420ea69437d8763866b7"
I20260926 14:56:20.768500 14599 raft_consensus.cc:3037] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:20.768500 14463 raft_consensus.cc:3037] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:20.769253 14463 raft_consensus.cc:2454] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ae81595f9464cadabc6fc7748af9c70 in term 1.
I20260926 14:56:20.769253 14599 raft_consensus.cc:2454] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ae81595f9464cadabc6fc7748af9c70 in term 1.
I20260926 14:56:20.769405 14263 leader_election.cc:302] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1445225627d3420ea69437d8763866b7, 8ae81595f9464cadabc6fc7748af9c70; no voters:
I20260926 14:56:20.769513 14681 raft_consensus.cc:2781] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:20.769627 14681 raft_consensus.cc:686] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [term 1 LEADER]: Becoming Leader. State: Replica: 8ae81595f9464cadabc6fc7748af9c70, State: Running, Role: LEADER
I20260926 14:56:20.769695 14681 consensus_queue.cc:237] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } } peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } }
I20260926 14:56:20.770375 13405 catalog_manager.cc:5495] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ae81595f9464cadabc6fc7748af9c70 (127.7.37.193). New cstate: current_term: 1 leader_uuid: "8ae81595f9464cadabc6fc7748af9c70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } health_report { overall_health: UNKNOWN } } }
I20260926 14:56:20.788895 14672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.786983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45668 (local address 127.7.37.193:42405)
0926 14:56:20.787028 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.787029 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.787038 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:56:20.787064 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.787065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.787079 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.787130 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:20.787226 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.787657 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.788355 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.788553 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.788601 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.788695 (+ 94us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:20.788774 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:20.788828 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.788840 (+ 12us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.788854 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 14:56:20.790272 14463 raft_consensus.cc:1261] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7 [term 1 FOLLOWER]: Refusing update from remote peer 8ae81595f9464cadabc6fc7748af9c70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:20.790297 14599 raft_consensus.cc:1261] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1 [term 1 FOLLOWER]: Refusing update from remote peer 8ae81595f9464cadabc6fc7748af9c70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:20.790571 14682 consensus_queue.cc:1035] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:20.790683 14681 consensus_queue.cc:1035] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:20.793746 14687 mvcc.cc:205] Tried to move back new op lower bound from 7333620042915065856 to 7333620042833190912. Current Snapshot: MvccSnapshot[applied={T|T < 7333620042915065856}]
I20260926 14:56:20.795454 13406 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:20.795543 13406 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:20.796653 13406 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e48c6b89a3847f2835be52b19a380ff: Sending DeleteTablet for 3 replicas of tablet b27ad5c536dd4a17a3ed21a231f4f625
I20260926 14:56:20.796939 14307 tablet_service.cc:1514] Processing DeleteTablet for tablet b27ad5c536dd4a17a3ed21a231f4f625 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:20 UTC) from {username='oryx', principal='oryx/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:44796
I20260926 14:56:20.796959 14579 tablet_service.cc:1514] Processing DeleteTablet for tablet b27ad5c536dd4a17a3ed21a231f4f625 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:20 UTC) from {username='oryx', principal='oryx/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:56824
I20260926 14:56:20.797194 14694 tablet_replica.cc:330] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1: stopping tablet replica
I20260926 14:56:20.797291 14694 raft_consensus.cc:2229] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:20.797302 14443 tablet_service.cc:1514] Processing DeleteTablet for tablet b27ad5c536dd4a17a3ed21a231f4f625 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:20 UTC) from {username='oryx', principal='oryx/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:44620
I20260926 14:56:20.797387 14694 raft_consensus.cc:2258] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:20.797907 14694 ts_tablet_manager.cc:1905] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:20.798007 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 13374
I20260926 14:56:20.799388 14694 ts_tablet_manager.cc:1918] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:56:20.799453 14694 log.cc:1196] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/b27ad5c536dd4a17a3ed21a231f4f625
I20260926 14:56:20.799690 14694 ts_tablet_manager.cc:1939] T b27ad5c536dd4a17a3ed21a231f4f625 P 356c4e7736584de99be4f7f0a33b2db1: Deleting consensus metadata
I20260926 14:56:20.800065 14696 tablet_replica.cc:330] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70: stopping tablet replica
I20260926 14:56:20.800158 14696 raft_consensus.cc:2229] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:56:20.800236 14696 raft_consensus.cc:2258] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 14:56:20.802659 14534 connection.cc:535] server connection from 127.0.0.1:56824 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 14:56:20.802846 14695 tablet_replica.cc:330] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7: stopping tablet replica
I20260926 14:56:20.802942 14695 raft_consensus.cc:2229] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:20.803041 14695 raft_consensus.cc:2258] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:20.803249 14696 ts_tablet_manager.cc:1905] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:20.804733 14696 ts_tablet_manager.cc:1918] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:56:20.804795 14696 log.cc:1196] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-0/wal/wals/b27ad5c536dd4a17a3ed21a231f4f625
I20260926 14:56:20.805042 14696 ts_tablet_manager.cc:1939] T b27ad5c536dd4a17a3ed21a231f4f625 P 8ae81595f9464cadabc6fc7748af9c70: Deleting consensus metadata
W20260926 14:56:20.805352 14263 connection.cc:441] server connection from 127.0.0.1:44796 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44796 (request call id 1) could send its response
I20260926 14:56:20.811297 14660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.804029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48850 (local address 127.7.37.252:39947)
0926 14:56:20.804065 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.804067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.804868 (+ 801us) server_negotiation.cc:407] Connection header received
0926 14:56:20.805499 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.805500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.805511 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.805555 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.806002 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.806604 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.807427 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.807704 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.807793 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.808951 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.808952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.808953 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.808959 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.810258 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.810537 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.810538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.810538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.810590 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.810771 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.810773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.810773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.810909 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.810998 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.811119 (+ 121us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.811198 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 14:56:20.811897 14152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:20.804121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57866 (local address 127.7.37.253:33917)
0926 14:56:20.804178 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:20.804180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:20.804736 (+ 556us) server_negotiation.cc:407] Connection header received
0926 14:56:20.804835 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:20.804836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:20.804849 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:20.804900 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:20.805452 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.806186 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.807098 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:20.807419 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:20.807490 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:20.808923 (+ 1433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:20.808924 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:20.808924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:20.808931 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:20.810539 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.810827 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.810829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.810829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.810878 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:20.811065 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:20.811066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:20.811067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:20.811185 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:20.811252 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:20.811622 (+ 370us) server_negotiation.cc:299] Negotiation successful
0926 14:56:20.811696 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 14:56:20.812073 14695 ts_tablet_manager.cc:1905] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:20.813285 14695 ts_tablet_manager.cc:1918] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:56:20.813335 14695 log.cc:1196] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/b27ad5c536dd4a17a3ed21a231f4f625
I20260926 14:56:20.813529 14695 ts_tablet_manager.cc:1939] T b27ad5c536dd4a17a3ed21a231f4f625 P 1445225627d3420ea69437d8763866b7: Deleting consensus metadata
W20260926 14:56:20.813802 14396 connection.cc:441] server connection from 127.0.0.1:44620 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44620 (request call id 1) could send its response
I20260926 14:56:21.793883 14704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:21.793609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.195:52479)
0926 14:56:21.793764 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:21.793830 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":70,"threads_started":1}
I20260926 14:56:21.793883 14705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:21.793646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.193:51417)
0926 14:56:21.793752 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:21.793831 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":49,"threads_started":1}
W20260926 14:56:21.794095 14646 heartbeater.cc:644] Failed to heartbeat to 127.7.37.254:34241 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
W20260926 14:56:21.794194 14374 heartbeater.cc:644] Failed to heartbeat to 127.7.37.254:34241 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
I20260926 14:56:21.794759 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:21.794545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.194:53615)
0926 14:56:21.794636 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:21.794702 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75,"thread_start_us":51,"threads_started":1}
W20260926 14:56:21.795027 14510 heartbeater.cc:644] Failed to heartbeat to 127.7.37.254:34241 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
I20260926 14:56:22.297289 14707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e48c6b89a3847f2835be52b19a380ff)
I20260926 14:56:22.297434 14707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } }
I20260926 14:56:22.297655 14707 leader_election.cc:288] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e48c6b89a3847f2835be52b19a380ff (127.7.37.254:34241), e3a46c25f46f453d9991a98b0f3ecff8 (127.7.37.253:33917)
I20260926 14:56:22.297669 14708 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e48c6b89a3847f2835be52b19a380ff)
I20260926 14:56:22.297782 14708 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } }
I20260926 14:56:22.297933 14709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:22.297740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:47492)
0926 14:56:22.297849 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:22.297892 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":49,"threads_started":1}
W20260926 14:56:22.298064 14179 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
I20260926 14:56:22.298255 14708 leader_election.cc:288] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e48c6b89a3847f2835be52b19a380ff (127.7.37.254:34241), 8f21b88955a146c9a566f880dafac879 (127.7.37.252:39947)
I20260926 14:56:22.298508 14713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:22.298176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:47498)
0926 14:56:22.298271 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:22.298320 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":48,"threads_started":1}
I20260926 14:56:22.298552 14709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:22.298469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:47510)
0926 14:56:22.298506 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:22.298526 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260926 14:56:22.298599 13991 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
W20260926 14:56:22.298629 14179 leader_election.cc:334] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0e48c6b89a3847f2835be52b19a380ff (127.7.37.254:34241): Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
I20260926 14:56:22.298988 14713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:22.298910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:47522)
0926 14:56:22.298945 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:22.298966 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260926 14:56:22.299067 13991 leader_election.cc:334] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0e48c6b89a3847f2835be52b19a380ff (127.7.37.254:34241): Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
I20260926 14:56:22.301131 14714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:22.298196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:39947 (local address 127.0.0.1:48866)
0926 14:56:22.298436 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:22.298445 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:22.298495 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:22.298727 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:22.298727 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:22.298729 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:22.298784 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:22.298785 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:22.299598 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:22.299599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:22.300399 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:22.300401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:22.300895 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:22.300896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:22.300949 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:22.300952 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:22.300994 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 14:56:22.301034 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":34,"threads_started":1}
I20260926 14:56:22.301124 14715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:22.298292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48866 (local address 127.7.37.252:39947)
0926 14:56:22.298594 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:22.298596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:22.298602 (+ 6us) server_negotiation.cc:407] Connection header received
0926 14:56:22.298622 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:22.298623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:22.298631 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:22.298687 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:22.298875 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:22.299537 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:22.300468 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:22.300826 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:22.300881 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:22.300891 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:22.301003 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 14:56:22.301035 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":87,"threads_started":1}
I20260926 14:56:22.301143 14711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:22.297814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57880 (local address 127.7.37.253:33917)
0926 14:56:22.297928 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:22.297930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:22.298066 (+ 136us) server_negotiation.cc:407] Connection header received
0926 14:56:22.298300 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:22.298301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:22.298311 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:22.298364 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:22.298716 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:22.299436 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:22.300474 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:22.300838 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:22.300901 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:22.300912 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:22.301014 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 14:56:22.301049 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
I20260926 14:56:22.301136 14710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:22.297771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:33917 (local address 127.0.0.1:57880)
0926 14:56:22.297944 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:22.297954 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:22.298218 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:22.298381 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:22.298382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:22.298384 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:22.298438 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:22.298440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:22.299528 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:22.299548 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:22.300404 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:22.300407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:22.300901 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:22.300901 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:22.300961 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:22.300964 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:22.301006 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 14:56:22.301046 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":1,"thread_start_us":39,"threads_started":1}
I20260926 14:56:22.301348 14202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8f21b88955a146c9a566f880dafac879" is_pre_election: true
I20260926 14:56:22.301348 14014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8f21b88955a146c9a566f880dafac879" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" is_pre_election: true
I20260926 14:56:22.301431 14202 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3a46c25f46f453d9991a98b0f3ecff8 in term 1.
I20260926 14:56:22.301431 14014 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f21b88955a146c9a566f880dafac879 in term 1.
I20260926 14:56:22.301579 13988 leader_election.cc:302] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f21b88955a146c9a566f880dafac879, e3a46c25f46f453d9991a98b0f3ecff8; no voters: 0e48c6b89a3847f2835be52b19a380ff
I20260926 14:56:22.301574 14176 leader_election.cc:302] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f21b88955a146c9a566f880dafac879, e3a46c25f46f453d9991a98b0f3ecff8; no voters: 0e48c6b89a3847f2835be52b19a380ff
I20260926 14:56:22.301654 14707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 14:56:22.301662 14708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 14:56:22.301684 14707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e48c6b89a3847f2835be52b19a380ff)
I20260926 14:56:22.301690 14708 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e48c6b89a3847f2835be52b19a380ff)
I20260926 14:56:22.301710 14708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 1 FOLLOWER]: Advancing to term 2
I20260926 14:56:22.301712 14707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 1 FOLLOWER]: Advancing to term 2
I20260926 14:56:22.302371 14707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } }
I20260926 14:56:22.302372 14708 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } }
I20260926 14:56:22.302467 14707 leader_election.cc:288] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [CANDIDATE]: Term 2 election: Requested vote from peers 0e48c6b89a3847f2835be52b19a380ff (127.7.37.254:34241), e3a46c25f46f453d9991a98b0f3ecff8 (127.7.37.253:33917)
I20260926 14:56:22.302469 14708 leader_election.cc:288] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [CANDIDATE]: Term 2 election: Requested vote from peers 0e48c6b89a3847f2835be52b19a380ff (127.7.37.254:34241), 8f21b88955a146c9a566f880dafac879 (127.7.37.252:39947)
I20260926 14:56:22.302618 14014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8f21b88955a146c9a566f880dafac879" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e3a46c25f46f453d9991a98b0f3ecff8"
I20260926 14:56:22.302645 14710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:22.302575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:47536)
0926 14:56:22.302601 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:22.302624 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
I20260926 14:56:22.302671 14714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:22.302549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:47528)
0926 14:56:22.302629 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:22.302652 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260926 14:56:22.302707 14014 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f21b88955a146c9a566f880dafac879 in current term 2: Already voted for candidate e3a46c25f46f453d9991a98b0f3ecff8 in this term.
I20260926 14:56:22.302857 14202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8f21b88955a146c9a566f880dafac879"
I20260926 14:56:22.302915 14710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:22.302846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:47558)
0926 14:56:22.302872 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:22.302895 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260926 14:56:22.302933 14202 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e3a46c25f46f453d9991a98b0f3ecff8 in current term 2: Already voted for candidate 8f21b88955a146c9a566f880dafac879 in this term.
I20260926 14:56:22.302949 14713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:22.302835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:47542)
0926 14:56:22.302904 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:22.302930 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 14:56:22.302973 14179 leader_election.cc:334] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0e48c6b89a3847f2835be52b19a380ff (127.7.37.254:34241): Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
I20260926 14:56:22.303001 14179 leader_election.cc:302] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f21b88955a146c9a566f880dafac879; no voters: 0e48c6b89a3847f2835be52b19a380ff, e3a46c25f46f453d9991a98b0f3ecff8
W20260926 14:56:22.303081 13991 leader_election.cc:334] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0e48c6b89a3847f2835be52b19a380ff (127.7.37.254:34241): Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
I20260926 14:56:22.303121 13991 leader_election.cc:302] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e3a46c25f46f453d9991a98b0f3ecff8; no voters: 0e48c6b89a3847f2835be52b19a380ff, 8f21b88955a146c9a566f880dafac879
I20260926 14:56:22.303185 14707 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260926 14:56:22.303189 14708 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260926 14:56:22.794662 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:22.794394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.195:60869)
0926 14:56:22.794580 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:22.794614 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
I20260926 14:56:22.794662 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:22.794476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.193:52929)
0926 14:56:22.794577 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:22.794614 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":41,"threads_started":1}
I20260926 14:56:22.795506 14719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:22.795296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.194:46383)
0926 14:56:22.795422 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:22.795456 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":44,"threads_started":1}
I20260926 14:56:23.795285 14720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.795073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.193:38045)
0926 14:56:23.795202 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:23.795236 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":56,"threads_started":1}
I20260926 14:56:23.795338 14721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.795073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.195:60907)
0926 14:56:23.795254 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:23.795291 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":58,"threads_started":1}
I20260926 14:56:23.796032 14722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.795868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.194:40351)
0926 14:56:23.795952 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:23.795982 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69,"thread_start_us":48,"threads_started":1}
I20260926 14:56:23.922075 14723 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:56:23.922168 14723 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } }
I20260926 14:56:23.922348 14723 leader_election.cc:288] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e48c6b89a3847f2835be52b19a380ff (127.7.37.254:34241), 8f21b88955a146c9a566f880dafac879 (127.7.37.252:39947)
I20260926 14:56:23.922560 14202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8f21b88955a146c9a566f880dafac879" is_pre_election: true
I20260926 14:56:23.922616 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.922442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:47754)
0926 14:56:23.922548 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:23.922582 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":70,"threads_started":1}
I20260926 14:56:23.922652 14202 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3a46c25f46f453d9991a98b0f3ecff8 in term 2.
I20260926 14:56:23.922796 13988 leader_election.cc:302] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f21b88955a146c9a566f880dafac879, e3a46c25f46f453d9991a98b0f3ecff8; no voters:
I20260926 14:56:23.922892 14723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260926 14:56:23.922930 14723 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:56:23.922947 14723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 2 FOLLOWER]: Advancing to term 3
I20260926 14:56:23.922996 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.922904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:47768)
0926 14:56:23.922938 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:23.922969 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260926 14:56:23.923067 13991 leader_election.cc:334] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0e48c6b89a3847f2835be52b19a380ff (127.7.37.254:34241): Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
I20260926 14:56:23.923640 14723 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } }
I20260926 14:56:23.923766 14723 leader_election.cc:288] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [CANDIDATE]: Term 3 election: Requested vote from peers 0e48c6b89a3847f2835be52b19a380ff (127.7.37.254:34241), 8f21b88955a146c9a566f880dafac879 (127.7.37.252:39947)
I20260926 14:56:23.923935 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.923843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:47782)
0926 14:56:23.923879 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:23.923910 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260926 14:56:23.923982 14202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8f21b88955a146c9a566f880dafac879"
I20260926 14:56:23.924054 14202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 2 FOLLOWER]: Advancing to term 3
I20260926 14:56:23.924216 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.924129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:47792)
0926 14:56:23.924174 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:23.924197 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 14:56:23.924310 13991 leader_election.cc:334] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0e48c6b89a3847f2835be52b19a380ff (127.7.37.254:34241): Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
I20260926 14:56:23.924717 14202 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3a46c25f46f453d9991a98b0f3ecff8 in term 3.
I20260926 14:56:23.924866 13988 leader_election.cc:302] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f21b88955a146c9a566f880dafac879, e3a46c25f46f453d9991a98b0f3ecff8; no voters: 0e48c6b89a3847f2835be52b19a380ff
I20260926 14:56:23.924950 14723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 14:56:23.925047 14723 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [term 3 LEADER]: Becoming Leader. State: Replica: e3a46c25f46f453d9991a98b0f3ecff8, State: Running, Role: LEADER
I20260926 14:56:23.925125 14723 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } }
I20260926 14:56:23.925467 14726 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3a46c25f46f453d9991a98b0f3ecff8. Latest consensus state: current_term: 3 leader_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } } }
I20260926 14:56:23.925555 14726 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:23.925707 14728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:56:23.926079 14728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:56:23.926131 14728 catalog_manager.cc:1259] Loaded cluster ID: dcfe92ec089b4fe7817234b268b008d2
I20260926 14:56:23.926149 14728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:56:23.926278 14728 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:23.926324 14728 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: Loaded TSK: 0
I20260926 14:56:23.926433 14728 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:56:23.944841 14003 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:56:23.946214 14003 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:56:23.947036 14202 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [term 3 FOLLOWER]: Refusing update from remote peer e3a46c25f46f453d9991a98b0f3ecff8: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260926 14:56:23.947042 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.946949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:47808)
0926 14:56:23.946990 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:23.947016 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260926 14:56:23.947204 14723 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260926 14:56:23.947340 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.947248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.0.0.1:47822)
0926 14:56:23.947288 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:23.947317 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 14:56:23.947412 13991 consensus_peers.cc:487] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 -> Peer 0e48c6b89a3847f2835be52b19a380ff (127.7.37.254:34241): Couldn't send request to peer 0e48c6b89a3847f2835be52b19a380ff. Status: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 14:56:23.947774 14730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3a46c25f46f453d9991a98b0f3ecff8. Latest consensus state: current_term: 3 leader_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } } }
I20260926 14:56:23.947918 14730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 14:56:23.948127 14729 mvcc.cc:205] Tried to move back new op lower bound from 7333620055845388288 to 7333620055757832192. Current Snapshot: MvccSnapshot[applied={T|T < 7333620055845388288}]
I20260926 14:56:23.948202 14726 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } } }
I20260926 14:56:23.948267 14726 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:23.948865 14731 mvcc.cc:205] Tried to move back new op lower bound from 7333620055845388288 to 7333620055757832192. Current Snapshot: MvccSnapshot[applied={T|T < 7333620055845388288 or (T in {7333620055845388288})}]
I20260926 14:56:23.948966 14730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0e48c6b89a3847f2835be52b19a380ff" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 34241 } } peers { permanent_uuid: "e3a46c25f46f453d9991a98b0f3ecff8" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 33917 } attrs { promote: false } } peers { permanent_uuid: "8f21b88955a146c9a566f880dafac879" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 39947 } attrs { promote: false } } }
I20260926 14:56:23.949045 14730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f21b88955a146c9a566f880dafac879 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 14:56:23.952977 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.949898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:42005 (local address 127.0.0.1:52740)
0926 14:56:23.950007 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:23.950016 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:23.950060 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:23.950366 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:23.950367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:23.950369 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:23.950412 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:23.950414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:23.951311 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:23.951312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:23.952102 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:23.952104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:23.952751 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:23.952752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:23.952815 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:23.952818 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:23.952871 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 14:56:23.952908 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":67,"threads_started":1}
I20260926 14:56:23.952991 14739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.950094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52740 (local address 127.7.37.194:42005)
0926 14:56:23.950196 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:23.950198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:23.950205 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:23.950227 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:23.950228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:23.950238 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:23.950340 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:23.950525 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:23.951242 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:23.952188 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:23.952678 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:23.952752 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:23.952765 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:23.952876 (+ 111us) server_negotiation.cc:299] Negotiation successful
0926 14:56:23.952909 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":60,"threads_started":1}
I20260926 14:56:23.953177 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.949847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:42405 (local address 127.0.0.1:45680)
0926 14:56:23.949873 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:23.949884 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:23.949932 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:23.950267 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:23.950268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:23.950269 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:23.950317 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:23.950319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:23.951250 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:23.951251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:23.952323 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:23.952325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:23.953013 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:23.953014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:23.953061 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:23.953063 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:56:23.953087 (+ 24us) client_negotiation.cc:240] Negotiation successful
0926 14:56:23.953115 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14}
I20260926 14:56:23.953179 14738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.949941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:42945 (local address 127.0.0.1:60082)
0926 14:56:23.950044 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:23.950052 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:23.950089 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:23.950411 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:23.950411 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:23.950413 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:23.950446 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:23.950447 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:23.951111 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:23.951111 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:23.952263 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:23.952265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:23.952972 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:23.952973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:23.953031 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:23.953036 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 14:56:23.953063 (+ 27us) client_negotiation.cc:240] Negotiation successful
0926 14:56:23.953101 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":71,"threads_started":1}
I20260926 14:56:23.953186 14443 tablet_service.cc:1467] Processing CreateTablet for tablet e726179ff42f4c57a35afad604c4de07 (DEFAULT_TABLE table=test-table [id=d8e9fb19b293489487b98059b399002a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:23.953315 14443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e726179ff42f4c57a35afad604c4de07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:23.953414 14740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.950194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60082 (local address 127.7.37.195:42945)
0926 14:56:23.950282 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:23.950284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:23.950290 (+ 6us) server_negotiation.cc:407] Connection header received
0926 14:56:23.950310 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:23.950310 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:23.950318 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:23.950501 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:23.950535 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:23.951040 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:23.952351 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:23.952886 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:23.953275 (+ 389us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:23.953289 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:23.953309 (+ 20us) server_negotiation.cc:299] Negotiation successful
0926 14:56:23.953347 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":46,"threads_started":1}
I20260926 14:56:23.953562 14579 tablet_service.cc:1467] Processing CreateTablet for tablet e726179ff42f4c57a35afad604c4de07 (DEFAULT_TABLE table=test-table [id=d8e9fb19b293489487b98059b399002a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:23.953666 14579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e726179ff42f4c57a35afad604c4de07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:23.953706 14736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.949907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45680 (local address 127.7.37.193:42405)
0926 14:56:23.950138 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:23.950140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:23.950147 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:23.950172 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:23.950173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:23.950182 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:23.950249 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:23.950404 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:23.951172 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:23.952407 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:23.952942 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:23.953542 (+ 600us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:23.953555 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:23.953572 (+ 17us) server_negotiation.cc:299] Negotiation successful
0926 14:56:23.953641 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":55,"threads_started":1}
I20260926 14:56:23.954684 14741 tablet_bootstrap.cc:492] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7: Bootstrap starting.
I20260926 14:56:23.954685 14742 tablet_bootstrap.cc:492] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1: Bootstrap starting.
I20260926 14:56:23.954761 14307 tablet_service.cc:1467] Processing CreateTablet for tablet e726179ff42f4c57a35afad604c4de07 (DEFAULT_TABLE table=test-table [id=d8e9fb19b293489487b98059b399002a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:23.954870 14307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e726179ff42f4c57a35afad604c4de07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:23.955024 14742 tablet_bootstrap.cc:654] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:23.955070 14741 tablet_bootstrap.cc:654] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:23.955541 14742 tablet_bootstrap.cc:492] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1: No bootstrap required, opened a new log
I20260926 14:56:23.955592 14742 ts_tablet_manager.cc:1397] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:23.955704 14741 tablet_bootstrap.cc:492] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7: No bootstrap required, opened a new log
I20260926 14:56:23.955775 14741 ts_tablet_manager.cc:1397] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:23.955719 14742 raft_consensus.cc:348] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } } peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } }
I20260926 14:56:23.955806 14742 raft_consensus.cc:374] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:23.955832 14742 raft_consensus.cc:729] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 356c4e7736584de99be4f7f0a33b2db1, State: Initialized, Role: FOLLOWER
I20260926 14:56:23.955883 14742 consensus_queue.cc:260] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } } peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } }
I20260926 14:56:23.955902 14741 raft_consensus.cc:348] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } } peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } }
I20260926 14:56:23.955966 14741 raft_consensus.cc:374] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:23.956007 14741 raft_consensus.cc:729] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1445225627d3420ea69437d8763866b7, State: Initialized, Role: FOLLOWER
I20260926 14:56:23.956049 14742 ts_tablet_manager.cc:1428] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:23.956049 14741 consensus_queue.cc:260] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } } peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } }
I20260926 14:56:23.956209 14741 ts_tablet_manager.cc:1428] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:23.956453 14721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.956364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.195:47681)
0926 14:56:23.956403 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:23.956428 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260926 14:56:23.956485 14722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.956363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.194:40073)
0926 14:56:23.956443 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:23.956464 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260926 14:56:23.956765 14721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.956694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.195:40571)
0926 14:56:23.956724 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:23.956742 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 14:56:23.956911 14511 heartbeater.cc:503] Master 127.7.37.253:33917 was elected leader, sending a full tablet report...
I20260926 14:56:23.957062 14722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.956969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.194:45695)
0926 14:56:23.957015 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:23.957038 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 14:56:23.957115 14647 heartbeater.cc:503] Master 127.7.37.253:33917 was elected leader, sending a full tablet report...
I20260926 14:56:23.957453 14746 tablet_bootstrap.cc:492] T e726179ff42f4c57a35afad604c4de07 P 8ae81595f9464cadabc6fc7748af9c70: Bootstrap starting.
I20260926 14:56:23.957772 14746 tablet_bootstrap.cc:654] T e726179ff42f4c57a35afad604c4de07 P 8ae81595f9464cadabc6fc7748af9c70: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:23.958135 14746 tablet_bootstrap.cc:492] T e726179ff42f4c57a35afad604c4de07 P 8ae81595f9464cadabc6fc7748af9c70: No bootstrap required, opened a new log
I20260926 14:56:23.958168 14746 ts_tablet_manager.cc:1397] T e726179ff42f4c57a35afad604c4de07 P 8ae81595f9464cadabc6fc7748af9c70: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260926 14:56:23.958236 14746 raft_consensus.cc:348] T e726179ff42f4c57a35afad604c4de07 P 8ae81595f9464cadabc6fc7748af9c70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } } peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } }
I20260926 14:56:23.958280 14746 raft_consensus.cc:374] T e726179ff42f4c57a35afad604c4de07 P 8ae81595f9464cadabc6fc7748af9c70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:23.958292 14746 raft_consensus.cc:729] T e726179ff42f4c57a35afad604c4de07 P 8ae81595f9464cadabc6fc7748af9c70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ae81595f9464cadabc6fc7748af9c70, State: Initialized, Role: FOLLOWER
I20260926 14:56:23.958330 14746 consensus_queue.cc:260] T e726179ff42f4c57a35afad604c4de07 P 8ae81595f9464cadabc6fc7748af9c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } } peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } }
I20260926 14:56:23.958463 14746 ts_tablet_manager.cc:1428] T e726179ff42f4c57a35afad604c4de07 P 8ae81595f9464cadabc6fc7748af9c70: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:23.958779 14720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.958687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.193:57695)
0926 14:56:23.958720 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:23.958752 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260926 14:56:23.958860 14375 heartbeater.cc:503] Master 127.7.37.253:33917 was elected leader, sending a full tablet report...
I20260926 14:56:23.959111 14720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:23.959033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.193:38341)
0926 14:56:23.959064 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:23.959088 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260926 14:56:24.116297 14648 tablet.cc:2369] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 14:56:24.117059 14376 tablet.cc:2369] T e726179ff42f4c57a35afad604c4de07 P 8ae81595f9464cadabc6fc7748af9c70: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:56:24.291188 14747 raft_consensus.cc:482] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:56:24.291287 14747 raft_consensus.cc:504] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } } peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } }
I20260926 14:56:24.291597 14747 leader_election.cc:288] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ae81595f9464cadabc6fc7748af9c70 (127.7.37.193:42405), 356c4e7736584de99be4f7f0a33b2db1 (127.7.37.195:42945)
I20260926 14:56:24.294364 14736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:24.291844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:46959 (local address 127.7.37.193:42405)
0926 14:56:24.291895 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:24.291897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:24.291907 (+ 10us) server_negotiation.cc:407] Connection header received
0926 14:56:24.291935 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:24.291936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:24.291949 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:24.292017 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:24.292169 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:24.292713 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:24.293689 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:24.294053 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:24.294103 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:24.294112 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:24.294264 (+ 152us) server_negotiation.cc:299] Negotiation successful
0926 14:56:24.294299 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 14:56:24.294378 14740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:24.292057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:44441 (local address 127.7.37.195:42945)
0926 14:56:24.292104 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:24.292106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:24.292114 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:24.292193 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:24.292194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:24.292205 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:24.292251 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:24.292393 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:24.292870 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:24.293725 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:24.294074 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:24.294138 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:24.294146 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:24.294286 (+ 140us) server_negotiation.cc:299] Negotiation successful
0926 14:56:24.294318 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 14:56:24.294395 14750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:24.291901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:42945 (local address 127.7.37.194:44441)
0926 14:56:24.292071 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:24.292081 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:24.292126 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:24.292266 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:24.292266 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:24.292268 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:24.292333 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:24.292335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:24.292941 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:24.292941 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:24.293644 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:24.293646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:24.294164 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:24.294165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:24.294214 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:24.294216 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:56:24.294273 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 14:56:24.294310 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
I20260926 14:56:24.294396 14722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:24.291779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:42405 (local address 127.7.37.194:46959)
0926 14:56:24.291820 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:24.291831 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:24.291881 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:24.292028 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:24.292029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:24.292032 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:24.292114 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:24.292116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:24.292792 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:24.292793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:24.293608 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:24.293610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:24.294130 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:24.294131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:24.294199 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:24.294202 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:24.294240 (+ 38us) client_negotiation.cc:240] Negotiation successful
0926 14:56:24.294275 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 14:56:24.294659 14599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e726179ff42f4c57a35afad604c4de07" candidate_uuid: "1445225627d3420ea69437d8763866b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "356c4e7736584de99be4f7f0a33b2db1" is_pre_election: true
I20260926 14:56:24.294747 14599 raft_consensus.cc:2454] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1445225627d3420ea69437d8763866b7 in term 0.
I20260926 14:56:24.294787 14327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e726179ff42f4c57a35afad604c4de07" candidate_uuid: "1445225627d3420ea69437d8763866b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae81595f9464cadabc6fc7748af9c70" is_pre_election: true
I20260926 14:56:24.294904 14327 raft_consensus.cc:2454] T e726179ff42f4c57a35afad604c4de07 P 8ae81595f9464cadabc6fc7748af9c70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1445225627d3420ea69437d8763866b7 in term 0.
I20260926 14:56:24.294904 14398 leader_election.cc:302] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1445225627d3420ea69437d8763866b7, 356c4e7736584de99be4f7f0a33b2db1; no voters:
I20260926 14:56:24.295019 14747 raft_consensus.cc:2781] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:56:24.295068 14747 raft_consensus.cc:482] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:56:24.295089 14747 raft_consensus.cc:3037] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:24.295727 14747 raft_consensus.cc:504] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } } peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } }
I20260926 14:56:24.295854 14747 leader_election.cc:288] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [CANDIDATE]: Term 1 election: Requested vote from peers 8ae81595f9464cadabc6fc7748af9c70 (127.7.37.193:42405), 356c4e7736584de99be4f7f0a33b2db1 (127.7.37.195:42945)
I20260926 14:56:24.296000 14327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e726179ff42f4c57a35afad604c4de07" candidate_uuid: "1445225627d3420ea69437d8763866b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae81595f9464cadabc6fc7748af9c70"
I20260926 14:56:24.296012 14599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e726179ff42f4c57a35afad604c4de07" candidate_uuid: "1445225627d3420ea69437d8763866b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "356c4e7736584de99be4f7f0a33b2db1"
I20260926 14:56:24.296075 14327 raft_consensus.cc:3037] T e726179ff42f4c57a35afad604c4de07 P 8ae81595f9464cadabc6fc7748af9c70 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:24.296075 14599 raft_consensus.cc:3037] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:24.296809 14599 raft_consensus.cc:2454] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1445225627d3420ea69437d8763866b7 in term 1.
I20260926 14:56:24.296809 14327 raft_consensus.cc:2454] T e726179ff42f4c57a35afad604c4de07 P 8ae81595f9464cadabc6fc7748af9c70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1445225627d3420ea69437d8763866b7 in term 1.
I20260926 14:56:24.296978 14398 leader_election.cc:302] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1445225627d3420ea69437d8763866b7, 8ae81595f9464cadabc6fc7748af9c70; no voters:
I20260926 14:56:24.297065 14747 raft_consensus.cc:2781] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:24.297195 14747 raft_consensus.cc:686] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [term 1 LEADER]: Becoming Leader. State: Replica: 1445225627d3420ea69437d8763866b7, State: Running, Role: LEADER
I20260926 14:56:24.297276 14747 consensus_queue.cc:237] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } } peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } }
I20260926 14:56:24.297515 14722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:24.297428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.194:48921)
0926 14:56:24.297458 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:24.297489 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260926 14:56:24.297891 14003 catalog_manager.cc:5495] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1445225627d3420ea69437d8763866b7 (127.7.37.194). New cstate: current_term: 1 leader_uuid: "1445225627d3420ea69437d8763866b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1445225627d3420ea69437d8763866b7" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42005 } health_report { overall_health: HEALTHY } } }
I20260926 14:56:24.304786 14739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:24.302633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52750 (local address 127.7.37.194:42005)
0926 14:56:24.302685 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:24.302687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:24.302694 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:24.302721 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:24.302722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:24.302756 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:24.302815 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:24.302915 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:24.303396 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:24.304152 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:24.304340 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:24.304388 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:24.304520 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:24.304616 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:24.304675 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:24.304710 (+ 35us) server_negotiation.cc:299] Negotiation successful
0926 14:56:24.304727 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 14:56:24.305637 14599 raft_consensus.cc:1261] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1 [term 1 FOLLOWER]: Refusing update from remote peer 1445225627d3420ea69437d8763866b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:24.305680 14327 raft_consensus.cc:1261] T e726179ff42f4c57a35afad604c4de07 P 8ae81595f9464cadabc6fc7748af9c70 [term 1 FOLLOWER]: Refusing update from remote peer 1445225627d3420ea69437d8763866b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:24.305861 14751 consensus_queue.cc:1035] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "356c4e7736584de99be4f7f0a33b2db1" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 42945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:24.305972 14747 consensus_queue.cc:1035] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ae81595f9464cadabc6fc7748af9c70" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 42405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:24.306571 14721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:24.306422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.195:50539)
0926 14:56:24.306513 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:24.306546 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260926 14:56:24.307351 14722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:24.307271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.194:59901)
0926 14:56:24.307305 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:24.307327 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260926 14:56:24.307628 14721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:24.307500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.195:39431)
0926 14:56:24.307580 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:24.307604 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260926 14:56:24.308670 14720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:24.308577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.193:53407)
0926 14:56:24.308613 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:24.308643 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260926 14:56:24.308801 14757 mvcc.cc:205] Tried to move back new op lower bound from 7333620057314279424 to 7333620057282125824. Current Snapshot: MvccSnapshot[applied={T|T < 7333620057314279424}]
I20260926 14:56:24.309051 14722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:24.308980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.194:54555)
0926 14:56:24.309005 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:24.309027 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260926 14:56:24.309320 14003 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:24.309397 14003 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:24.309722 14720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:24.309308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:34241 (local address 127.7.37.193:33059)
0926 14:56:24.309668 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:24.309697 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344}
I20260926 14:56:24.310382 14003 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e3a46c25f46f453d9991a98b0f3ecff8: Sending DeleteTablet for 3 replicas of tablet e726179ff42f4c57a35afad604c4de07
I20260926 14:56:24.310639 14443 tablet_service.cc:1514] Processing DeleteTablet for tablet e726179ff42f4c57a35afad604c4de07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:24 UTC) from {username='oryx', principal='oryx/127.7.37.253@KRBTEST.COM'} at 127.0.0.1:52740
I20260926 14:56:24.310649 14307 tablet_service.cc:1514] Processing DeleteTablet for tablet e726179ff42f4c57a35afad604c4de07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:24 UTC) from {username='oryx', principal='oryx/127.7.37.253@KRBTEST.COM'} at 127.0.0.1:45680
I20260926 14:56:24.310689 14579 tablet_service.cc:1514] Processing DeleteTablet for tablet e726179ff42f4c57a35afad604c4de07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:24 UTC) from {username='oryx', principal='oryx/127.7.37.253@KRBTEST.COM'} at 127.0.0.1:60082
I20260926 14:56:24.311058 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 14244
I20260926 14:56:24.310930 14767 tablet_replica.cc:330] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1: stopping tablet replica
I20260926 14:56:24.311144 14767 raft_consensus.cc:2229] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:24.311182 14767 raft_consensus.cc:2258] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:24.311268 14765 tablet_replica.cc:330] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7: stopping tablet replica
I20260926 14:56:24.311302 14765 raft_consensus.cc:2229] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:56:24.311355 14765 raft_consensus.cc:2258] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:24.311718 14767 ts_tablet_manager.cc:1905] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:24.312862 14767 ts_tablet_manager.cc:1918] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:56:24.312868 14765 ts_tablet_manager.cc:1905] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:24.312974 14767 log.cc:1196] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/e726179ff42f4c57a35afad604c4de07
I20260926 14:56:24.313254 14767 ts_tablet_manager.cc:1939] T e726179ff42f4c57a35afad604c4de07 P 356c4e7736584de99be4f7f0a33b2db1: Deleting consensus metadata
I20260926 14:56:24.313556 13990 catalog_manager.cc:4841] TS 356c4e7736584de99be4f7f0a33b2db1 (127.7.37.195:42945): tablet e726179ff42f4c57a35afad604c4de07 (table test-table [id=d8e9fb19b293489487b98059b399002a]) successfully deleted
I20260926 14:56:24.314177 14765 ts_tablet_manager.cc:1918] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:56:24.314224 14765 log.cc:1196] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/e726179ff42f4c57a35afad604c4de07
I20260926 14:56:24.314415 14765 ts_tablet_manager.cc:1939] T e726179ff42f4c57a35afad604c4de07 P 1445225627d3420ea69437d8763866b7: Deleting consensus metadata
I20260926 14:56:24.314667 13991 catalog_manager.cc:4841] TS 1445225627d3420ea69437d8763866b7 (127.7.37.194:42005): tablet e726179ff42f4c57a35afad604c4de07 (table test-table [id=d8e9fb19b293489487b98059b399002a]) successfully deleted
W20260926 14:56:24.319468 14738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:24.319361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:42405 (local address 127.0.0.1:45684)
0926 14:56:24.319399 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:24.319424 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.193:42405: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":21}
W20260926 14:56:24.319578 13990 catalog_manager.cc:4584] TS 8ae81595f9464cadabc6fc7748af9c70 (127.7.37.193:42405): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e726179ff42f4c57a35afad604c4de07: Network error: Client connection negotiation failed: client connection to 127.7.37.193:42405: connect: Connection reset by peer (error 104)
I20260926 14:56:24.319623 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 14380
I20260926 14:56:24.323829 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 14516
I20260926 14:56:24.327273 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 13870
I20260926 14:56:24.330888 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 14058
2026-09-26T14:56:24Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6285 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:56:24 dist-test-slave-f9xb krb5kdc[14770](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:56:24 dist-test-slave-f9xb krb5kdc[14770](info): set up 2 sockets
Sep 26 14:56:24 dist-test-slave-f9xb krb5kdc[14770](info): commencing operation
krb5kdc: starting...
W20260926 14:56:25.275684 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:56:25 dist-test-slave-f9xb krb5kdc[14770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:56:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:25Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.254.keytab.
Entry for principal oryx/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.254.keytab.
I20260926 14:56:25.378319 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36783
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.254.keytab
--principal=oryx/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:35427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:36783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:25.453022 14783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:25.453193 14783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:25.453212 14783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:25.454262 14783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:25.454294 14783 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 14:56:25.454305 14783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:25.454317 14783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:25.454327 14783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:25.454341 14783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:56:25.455718 14783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:35427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:36783
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.254.keytab
--principal=oryx/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:25.455912 14783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:25.456059 14783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:25.460177 14788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:25.460191 14789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:25.460230 14791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:25.460255 14783 server_base.cc:1034] running on GCE node
I20260926 14:56:25.460574 14783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:25.460824 14783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:25.461959 14783 hybrid_clock.cc:648] HybridClock initialized: now 1790434585461945 us; error 30 us; skew 500 ppm
Sep 26 14:56:25 dist-test-slave-f9xb krb5kdc[14770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434585, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:25.464174 14783 init.cc:377] Logged in from keytab as oryx/127.7.37.254@KRBTEST.COM (short username oryx)
I20260926 14:56:25.464618 14783 webserver.cc:460] Webserver started at http://127.7.37.254:37929/ using document root <none> and password file <none>
I20260926 14:56:25.464855 14783 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:25.464907 14783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:25.465008 14783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:25.465909 14783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "ba9985f1481347e8bf5ee0656f4a6f6a"
format_stamp: "Formatted at 2026-09-26 14:56:25 on dist-test-slave-f9xb"
server_key: "999fb8cb0b9624f085bb33d0bd6d701f"
server_key_iv: "b88a9d4e5c5f8dcefc1cab5dbaa141f3"
server_key_version: "encryptionkey@0"
I20260926 14:56:25.466210 14783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "ba9985f1481347e8bf5ee0656f4a6f6a"
format_stamp: "Formatted at 2026-09-26 14:56:25 on dist-test-slave-f9xb"
server_key: "999fb8cb0b9624f085bb33d0bd6d701f"
server_key_iv: "b88a9d4e5c5f8dcefc1cab5dbaa141f3"
server_key_version: "encryptionkey@0"
I20260926 14:56:25.467326 14783 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 14:56:25.467924 14798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:25.468072 14783 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:25.468145 14783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "ba9985f1481347e8bf5ee0656f4a6f6a"
format_stamp: "Formatted at 2026-09-26 14:56:25 on dist-test-slave-f9xb"
server_key: "999fb8cb0b9624f085bb33d0bd6d701f"
server_key_iv: "b88a9d4e5c5f8dcefc1cab5dbaa141f3"
server_key_version: "encryptionkey@0"
I20260926 14:56:25.468209 14783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:25.483923 14783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:25.485455 14783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:25.485574 14783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:25.489554 14783 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:36783
I20260926 14:56:25.489609 14860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:36783 every 8 connection(s)
I20260926 14:56:25.489873 14783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:56:25.490363 14861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:25.491528 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 14783
I20260926 14:56:25.491609 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:56:25.491808 7319 external_mini_cluster.cc:1428] Setting key b3b592e121bc0edaaf9119fa97475a35
I20260926 14:56:25.492503 14861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a: Bootstrap starting.
I20260926 14:56:25.493005 14861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:25.493248 14861 log.cc:824] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:25.493896 14861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a: No bootstrap required, opened a new log
I20260926 14:56:25.495882 14861 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9985f1481347e8bf5ee0656f4a6f6a" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36783 } }
I20260926 14:56:25.496038 14861 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:25.496070 14861 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba9985f1481347e8bf5ee0656f4a6f6a, State: Initialized, Role: FOLLOWER
I20260926 14:56:25.496183 14861 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9985f1481347e8bf5ee0656f4a6f6a" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36783 } }
I20260926 14:56:25.496265 14861 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:56:25.496304 14861 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:56:25.496340 14861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:25.496765 14861 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9985f1481347e8bf5ee0656f4a6f6a" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36783 } }
I20260926 14:56:25.496845 14861 leader_election.cc:302] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba9985f1481347e8bf5ee0656f4a6f6a; no voters:
I20260926 14:56:25.496975 14861 leader_election.cc:288] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:56:25.497022 14866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:25.497164 14866 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a [term 1 LEADER]: Becoming Leader. State: Replica: ba9985f1481347e8bf5ee0656f4a6f6a, State: Running, Role: LEADER
I20260926 14:56:25.497184 14861 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:25.497262 14866 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9985f1481347e8bf5ee0656f4a6f6a" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36783 } }
I20260926 14:56:25.497638 14868 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba9985f1481347e8bf5ee0656f4a6f6a. Latest consensus state: current_term: 1 leader_uuid: "ba9985f1481347e8bf5ee0656f4a6f6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9985f1481347e8bf5ee0656f4a6f6a" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36783 } } }
I20260926 14:56:25.497725 14868 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:25.497653 14867 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba9985f1481347e8bf5ee0656f4a6f6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9985f1481347e8bf5ee0656f4a6f6a" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36783 } } }
I20260926 14:56:25.497936 14867 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:25.498036 14874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 26 14:56:25 dist-test-slave-f9xb krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM
I20260926 14:56:25.498466 14874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:56:25.501611 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.492247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54226 (local address 127.7.37.254:36783)
0926 14:56:25.492411 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:25.492418 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:25.492439 (+ 21us) server_negotiation.cc:407] Connection header received
0926 14:56:25.492820 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:25.492827 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:25.492869 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:25.492983 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:25.493605 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.494952 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.495630 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.496225 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.496331 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.498827 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:25.498830 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:25.498830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:25.498839 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:25.500321 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:25.500680 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:25.500682 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:25.500682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:25.500764 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:25.500941 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:25.500941 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:25.500941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:25.501124 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:25.501208 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:25.501348 (+ 140us) server_negotiation.cc:299] Negotiation successful
0926 14:56:25.501434 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260926 14:56:25.504623 14874 catalog_manager.cc:1347] Generated new cluster ID: f38abdd9e21346c7997f2519d2a7e818
I20260926 14:56:25.504673 14874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:56:25.519161 14874 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:56:25.519622 14874 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:25.524042 14874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a: Generated new TSK 0
I20260926 14:56:25.524174 14874 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.193.keytab.
Entry for principal oryx/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.193.keytab.
I20260926 14:56:25.549928 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36783
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.193.keytab
--principal=oryx/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:35427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:25.643327 14887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:25.643510 14887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:25.643544 14887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:25.644694 14887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:25.644733 14887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:25.644752 14887 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 14:56:25.644776 14887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:56:25.646200 14887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:35427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.193.keytab
--principal=oryx/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.37.254:36783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:25.646390 14887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:25.646580 14887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:25.650936 14895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:25.650940 14893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:25.650926 14892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:25.651258 14887 server_base.cc:1034] running on GCE node
I20260926 14:56:25.651451 14887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:25.651647 14887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:25.652779 14887 hybrid_clock.cc:648] HybridClock initialized: now 1790434585652765 us; error 31 us; skew 500 ppm
Sep 26 14:56:25 dist-test-slave-f9xb krb5kdc[14770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434585, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:25.654973 14887 init.cc:377] Logged in from keytab as oryx/127.7.37.193@KRBTEST.COM (short username oryx)
I20260926 14:56:25.655342 14887 webserver.cc:460] Webserver started at http://127.7.37.193:35197/ using document root <none> and password file <none>
I20260926 14:56:25.655501 14887 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:25.655537 14887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:25.655619 14887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:25.656519 14887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "df829259b2534be29a2257d356c85a9d"
format_stamp: "Formatted at 2026-09-26 14:56:25 on dist-test-slave-f9xb"
server_key: "f460419afc5781514ca2e0c1deec7d5a"
server_key_iv: "6bbd2bea7ceacaf62d0002f97ba6fcaf"
server_key_version: "encryptionkey@0"
I20260926 14:56:25.656786 14887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "df829259b2534be29a2257d356c85a9d"
format_stamp: "Formatted at 2026-09-26 14:56:25 on dist-test-slave-f9xb"
server_key: "f460419afc5781514ca2e0c1deec7d5a"
server_key_iv: "6bbd2bea7ceacaf62d0002f97ba6fcaf"
server_key_version: "encryptionkey@0"
I20260926 14:56:25.657869 14887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:25.658454 14902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:25.658607 14887 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:25.658685 14887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "df829259b2534be29a2257d356c85a9d"
format_stamp: "Formatted at 2026-09-26 14:56:25 on dist-test-slave-f9xb"
server_key: "f460419afc5781514ca2e0c1deec7d5a"
server_key_iv: "6bbd2bea7ceacaf62d0002f97ba6fcaf"
server_key_version: "encryptionkey@0"
I20260926 14:56:25.658743 14887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:25.667003 14887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:25.668984 14887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:25.669088 14887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:25.669516 14887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:25.669553 14887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:25.669574 14887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:25.669591 14887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 14:56:25 dist-test-slave-f9xb krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434585, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM
I20260926 14:56:25.676190 14887 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:35495
I20260926 14:56:25.676348 15017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:35495 every 8 connection(s)
I20260926 14:56:25.676548 14887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:56:25.678500 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.669813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:56303 (local address 127.7.37.254:36783)
0926 14:56:25.669870 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:25.669872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:25.670047 (+ 175us) server_negotiation.cc:407] Connection header received
0926 14:56:25.670457 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:25.670459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:25.670472 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:25.670557 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:25.671233 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.671959 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.673374 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.673656 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.673752 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.676043 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:25.676045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:25.676045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:25.676052 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:25.677433 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:25.677782 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:25.677783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:25.677783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:25.677830 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:25.678006 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:25.678007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:25.678007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:25.678096 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:25.678140 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:25.678386 (+ 246us) server_negotiation.cc:299] Negotiation successful
0926 14:56:25.678442 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 14:56:25.678645 14911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.669810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36783 (local address 127.7.37.193:56303)
0926 14:56:25.669937 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:25.669971 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:25.670369 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:25.670600 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:25.670603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:25.670856 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:25.671139 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:25.671145 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.672052 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.672053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:25.673288 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:25.673290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.673760 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.673761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:25.673910 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.674372 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:25.674381 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:25.675952 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:25.677502 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:25.677506 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:25.677508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:25.677719 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:25.677875 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:25.677876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:25.677876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:25.677962 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:25.678169 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:25.678175 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:25.678278 (+ 103us) client_negotiation.cc:769] Sending connection context
0926 14:56:25.678369 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 14:56:25.678473 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":71,"threads_started":1}
I20260926 14:56:25.679085 15018 heartbeater.cc:347] Connected to a master server at 127.7.37.254:36783
I20260926 14:56:25.679150 15018 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:25.679286 15018 heartbeater.cc:511] Master 127.7.37.254:36783 requested a full tablet report, sending...
I20260926 14:56:25.679768 14814 ts_manager.cc:194] Registered new tserver with Master: df829259b2534be29a2257d356c85a9d (127.7.37.193:35495)
I20260926 14:56:25.680586 14814 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:56303
I20260926 14:56:25.684095 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 14887
I20260926 14:56:25.684208 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:56:25.684412 7319 external_mini_cluster.cc:1428] Setting key de4a6bb0d67dab7b6688caebf4c65770
I20260926 14:56:25.684593 15024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.679910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36783 (local address 127.7.37.193:52877)
0926 14:56:25.680021 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:25.680031 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:25.680086 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:25.680192 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:25.680193 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:25.680370 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:25.680421 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:25.680422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.681063 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.681064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:25.681686 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:25.681688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.681963 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.681964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:25.682004 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.682359 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:25.682363 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:25.682597 (+ 234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:25.683686 (+ 1089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:25.683687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:25.683687 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:25.683880 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:25.684062 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:25.684062 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:25.684063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:25.684105 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:25.684345 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:25.684346 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:25.684380 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 14:56:25.684427 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 14:56:25.684502 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":18,"thread_start_us":58,"threads_started":1}
I20260926 14:56:25.684612 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.679952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:52877 (local address 127.7.37.254:36783)
0926 14:56:25.679986 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:25.679987 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:25.680074 (+ 87us) server_negotiation.cc:407] Connection header received
0926 14:56:25.680137 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:25.680137 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:25.680144 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:25.680180 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:25.680473 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.680979 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.681743 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.681917 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.681956 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.682652 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:25.682653 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:25.682653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:25.682657 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:25.683626 (+ 969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:25.683948 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:25.683949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:25.683949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:25.684003 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:25.684141 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:25.684141 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:25.684142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:25.684255 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:25.684312 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:25.684466 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 14:56:25.684534 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for oryx/127.7.37.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.37.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.194.keytab.
Entry for principal oryx/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.194.keytab.
I20260926 14:56:25.705916 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36783
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.194.keytab
--principal=oryx/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:35427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:25.780412 15027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:25.780575 15027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:25.780596 15027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:25.781680 15027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:25.781719 15027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:25.781739 15027 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 14:56:25.781769 15027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:56:25.783210 15027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:35427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.194.keytab
--principal=oryx/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.37.254:36783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:25.783408 15027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:25.783594 15027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:25.787853 15033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:25.787842 15032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:25.787967 15027 server_base.cc:1034] running on GCE node
W20260926 14:56:25.787832 15035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:25.788208 15027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:25.788388 15027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:25.789521 15027 hybrid_clock.cc:648] HybridClock initialized: now 1790434585789510 us; error 26 us; skew 500 ppm
Sep 26 14:56:25 dist-test-slave-f9xb krb5kdc[14770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434585, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:25.791678 15027 init.cc:377] Logged in from keytab as oryx/127.7.37.194@KRBTEST.COM (short username oryx)
I20260926 14:56:25.792061 15027 webserver.cc:460] Webserver started at http://127.7.37.194:39709/ using document root <none> and password file <none>
I20260926 14:56:25.792244 15027 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:25.792289 15027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:25.792402 15027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:25.793269 15027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "0e1f78bf01a6418da2dded31464d28ac"
format_stamp: "Formatted at 2026-09-26 14:56:25 on dist-test-slave-f9xb"
server_key: "c351ba10fa52f86cf9362e4fb8882129"
server_key_iv: "070cdcac5e79c7a9d321d97536a0861c"
server_key_version: "encryptionkey@0"
I20260926 14:56:25.793548 15027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "0e1f78bf01a6418da2dded31464d28ac"
format_stamp: "Formatted at 2026-09-26 14:56:25 on dist-test-slave-f9xb"
server_key: "c351ba10fa52f86cf9362e4fb8882129"
server_key_iv: "070cdcac5e79c7a9d321d97536a0861c"
server_key_version: "encryptionkey@0"
I20260926 14:56:25.794617 15027 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:25.795233 15042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:25.795373 15027 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:25.795444 15027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "0e1f78bf01a6418da2dded31464d28ac"
format_stamp: "Formatted at 2026-09-26 14:56:25 on dist-test-slave-f9xb"
server_key: "c351ba10fa52f86cf9362e4fb8882129"
server_key_iv: "070cdcac5e79c7a9d321d97536a0861c"
server_key_version: "encryptionkey@0"
I20260926 14:56:25.795499 15027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:25.815595 15027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:25.817476 15027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:25.817595 15027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:25.817996 15027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:25.818032 15027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:25.818055 15027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:25.818070 15027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 14:56:25 dist-test-slave-f9xb krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434585, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM
I20260926 14:56:25.824370 15027 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:37885
I20260926 14:56:25.824438 15157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:37885 every 8 connection(s)
I20260926 14:56:25.824646 15027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:56:25.826761 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.818355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:49141 (local address 127.7.37.254:36783)
0926 14:56:25.818425 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:25.818427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:25.818592 (+ 165us) server_negotiation.cc:407] Connection header received
0926 14:56:25.819117 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:25.819118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:25.819128 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:25.819171 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:25.819959 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.820476 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.821896 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.822183 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.822270 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.824336 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:25.824337 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:25.824338 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:25.824345 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:25.825638 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:25.825962 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:25.825964 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:25.825964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:25.826017 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:25.826214 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:25.826215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:25.826215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:25.826324 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:25.826376 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:25.826604 (+ 228us) server_negotiation.cc:299] Negotiation successful
0926 14:56:25.826676 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 14:56:25.826885 15051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.818326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36783 (local address 127.7.37.194:49141)
0926 14:56:25.818469 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:25.818509 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:25.819017 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:25.819240 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:25.819244 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:25.819569 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:25.819855 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:25.819861 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.820553 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.820554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:25.821809 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:25.821811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.822292 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.822293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:25.822443 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.822881 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:25.822890 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:25.824250 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:25.825706 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:25.825712 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:25.825714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:25.825903 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:25.826066 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:25.826066 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:25.826067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:25.826161 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:25.826375 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:25.826381 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:25.826497 (+ 116us) client_negotiation.cc:769] Sending connection context
0926 14:56:25.826576 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 14:56:25.826704 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
I20260926 14:56:25.827229 15158 heartbeater.cc:347] Connected to a master server at 127.7.37.254:36783
I20260926 14:56:25.827298 15158 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:25.827432 15158 heartbeater.cc:511] Master 127.7.37.254:36783 requested a full tablet report, sending...
I20260926 14:56:25.827726 14815 ts_manager.cc:194] Registered new tserver with Master: 0e1f78bf01a6418da2dded31464d28ac (127.7.37.194:37885)
I20260926 14:56:25.828202 14815 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:49141
I20260926 14:56:25.829406 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 15027
I20260926 14:56:25.829504 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:56:25.829665 7319 external_mini_cluster.cc:1428] Setting key e97b903ad078d246d31c046592a20b03
I20260926 14:56:25.832906 15164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.828042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36783 (local address 127.7.37.194:60921)
0926 14:56:25.828174 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:25.828183 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:25.828241 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:25.828348 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:25.828349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:25.828488 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:25.828529 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:25.828530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.829100 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.829101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:25.829670 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:25.829672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.829984 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.829985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:25.830045 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.830398 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:25.830402 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:25.830635 (+ 233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:25.832100 (+ 1465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:25.832101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:25.832101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:25.832245 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:25.832415 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:25.832416 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:25.832416 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:25.832461 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:25.832708 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:25.832708 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:25.832739 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 14:56:25.832781 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 14:56:25.832839 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":56,"threads_started":1}
I20260926 14:56:25.832937 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.828101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:60921 (local address 127.7.37.254:36783)
0926 14:56:25.828136 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:25.828138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:25.828238 (+ 100us) server_negotiation.cc:407] Connection header received
0926 14:56:25.828284 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:25.828285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:25.828295 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:25.828345 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:25.828593 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.829024 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.829732 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.829931 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.829979 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.830706 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:25.830707 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:25.830707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:25.830713 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:25.832030 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:25.832311 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:25.832311 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:25.832312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:25.832362 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:25.832507 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:25.832508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:25.832508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:25.832658 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:25.832718 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:25.832802 (+ 84us) server_negotiation.cc:299] Negotiation successful
0926 14:56:25.832863 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for oryx/127.7.37.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.37.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.195.keytab.
Entry for principal oryx/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.195.keytab.
I20260926 14:56:25.850339 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36783
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.195.keytab
--principal=oryx/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:35427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:25.925448 15167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:25.925638 15167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:25.925660 15167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:25.926764 15167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:25.926800 15167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:25.926821 15167 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 14:56:25.926851 15167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:56:25.928354 15167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:35427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.195.keytab
--principal=oryx/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.37.254:36783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:25.928543 15167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:25.928725 15167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:25.933102 15173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:25.933096 15172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:25.933127 15167 server_base.cc:1034] running on GCE node
W20260926 14:56:25.933092 15175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:25.933507 15167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:25.933701 15167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:25.934834 15167 hybrid_clock.cc:648] HybridClock initialized: now 1790434585934823 us; error 27 us; skew 500 ppm
Sep 26 14:56:25 dist-test-slave-f9xb krb5kdc[14770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434585, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:25.937141 15167 init.cc:377] Logged in from keytab as oryx/127.7.37.195@KRBTEST.COM (short username oryx)
I20260926 14:56:25.937510 15167 webserver.cc:460] Webserver started at http://127.7.37.195:41241/ using document root <none> and password file <none>
I20260926 14:56:25.937692 15167 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:25.937731 15167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:25.937836 15167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:25.938715 15167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "53ad97f8c2524129bbe02a47e291ee98"
format_stamp: "Formatted at 2026-09-26 14:56:25 on dist-test-slave-f9xb"
server_key: "a3a4734e13c684c01cdc1437b4c45717"
server_key_iv: "f077c9534ea9707d03b3053304d070cf"
server_key_version: "encryptionkey@0"
I20260926 14:56:25.939028 15167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "53ad97f8c2524129bbe02a47e291ee98"
format_stamp: "Formatted at 2026-09-26 14:56:25 on dist-test-slave-f9xb"
server_key: "a3a4734e13c684c01cdc1437b4c45717"
server_key_iv: "f077c9534ea9707d03b3053304d070cf"
server_key_version: "encryptionkey@0"
I20260926 14:56:25.940187 15167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:25.940830 15182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:25.941007 15167 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:25.941087 15167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "53ad97f8c2524129bbe02a47e291ee98"
format_stamp: "Formatted at 2026-09-26 14:56:25 on dist-test-slave-f9xb"
server_key: "a3a4734e13c684c01cdc1437b4c45717"
server_key_iv: "f077c9534ea9707d03b3053304d070cf"
server_key_version: "encryptionkey@0"
I20260926 14:56:25.941138 15167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:25.950014 15167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:25.951596 15167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:25.951707 15167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:25.952144 15167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:25.952185 15167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:25.952209 15167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:25.952229 15167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 14:56:25 dist-test-slave-f9xb krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434585, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM
I20260926 14:56:25.958608 15167 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:41361
I20260926 14:56:25.958673 15297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:41361 every 8 connection(s)
I20260926 14:56:25.958905 15167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:56:25.961357 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.952540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:43777 (local address 127.7.37.254:36783)
0926 14:56:25.952603 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:25.952605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:25.952736 (+ 131us) server_negotiation.cc:407] Connection header received
0926 14:56:25.953107 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:25.953109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:25.953122 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:25.953213 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:25.953861 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.954589 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.956023 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.956308 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.956393 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.958762 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:25.958763 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:25.958763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:25.958770 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:25.960182 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:25.960517 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:25.960518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:25.960518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:25.960568 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:25.960765 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:25.960766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:25.960766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:25.960900 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:25.960956 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:25.961194 (+ 238us) server_negotiation.cc:299] Negotiation successful
0926 14:56:25.961269 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 14:56:25.961500 15191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.952499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36783 (local address 127.7.37.195:43777)
0926 14:56:25.952627 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:25.952660 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:25.953029 (+ 369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:25.953247 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:25.953249 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:25.953500 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:25.953770 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:25.953775 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.954687 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.954688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:25.955924 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:25.955926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.956423 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.956424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:25.956525 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.956969 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:25.956978 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:25.958679 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:25.960249 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:25.960254 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:25.960256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:25.960455 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:25.960619 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:25.960619 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:25.960620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:25.960714 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:25.960961 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:25.960966 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:25.961080 (+ 114us) client_negotiation.cc:769] Sending connection context
0926 14:56:25.961160 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 14:56:25.961287 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":70,"threads_started":1}
I20260926 14:56:25.961858 15298 heartbeater.cc:347] Connected to a master server at 127.7.37.254:36783
I20260926 14:56:25.961930 15298 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:25.962098 15298 heartbeater.cc:511] Master 127.7.37.254:36783 requested a full tablet report, sending...
I20260926 14:56:25.962423 14815 ts_manager.cc:194] Registered new tserver with Master: 53ad97f8c2524129bbe02a47e291ee98 (127.7.37.195:41361)
I20260926 14:56:25.962869 14815 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:43777
I20260926 14:56:25.963892 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 15167
I20260926 14:56:25.963989 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:56:25.964146 7319 external_mini_cluster.cc:1428] Setting key 898e596439ecaeea36f63e1d9eee7d3d
I20260926 14:56:25.964847 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 14:56:25.967690 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.962729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:47553 (local address 127.7.37.254:36783)
0926 14:56:25.962764 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:25.962766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:25.962810 (+ 44us) server_negotiation.cc:407] Connection header received
0926 14:56:25.962868 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:25.962869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:25.962875 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:25.962916 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:25.963202 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.963656 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.964474 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.964722 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.964783 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.965523 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:25.965524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:25.965525 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:25.965530 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:25.966756 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:25.967111 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:25.967112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:25.967112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:25.967156 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:25.967326 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:25.967327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:25.967327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:25.967407 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:25.967446 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:25.967583 (+ 137us) server_negotiation.cc:299] Negotiation successful
0926 14:56:25.967632 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:56:25.967799 15304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.962662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36783 (local address 127.7.37.195:47553)
0926 14:56:25.962760 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:25.962769 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:25.962820 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:25.962921 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:25.962922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:25.963098 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:25.963139 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:25.963141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.963739 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.963761 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:25.964384 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:25.964386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.964791 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.964792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:25.964856 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.965203 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:25.965206 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:25.965453 (+ 247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:25.966824 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:25.966825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:25.966825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:25.967046 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:25.967205 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:25.967206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:25.967206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:25.967272 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:25.967454 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:25.967455 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:25.967495 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 14:56:25.967562 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 14:56:25.967645 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":55,"threads_started":1}
Sep 26 14:56:25 dist-test-slave-f9xb krb5kdc[14770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434585, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 14:56:25 dist-test-slave-f9xb krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:25 dist-test-slave-f9xb krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 14:56:25.984163 14864 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:25.979607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54240 (local address 127.7.37.254:36783)
0926 14:56:25.979651 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:25.979652 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:25.979659 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:25.979726 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:25.979727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:25.979735 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:25.979792 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:25.980300 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.980727 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.981407 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.981582 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.981630 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.984108 (+ 2478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54240: BlockingRecv error: recv got EOF from 127.0.0.1:54240 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Sep 26 14:56:25 dist-test-slave-f9xb krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434585, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM
I20260926 14:56:25.990775 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.984706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54248 (local address 127.7.37.254:36783)
0926 14:56:25.984740 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:25.984741 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:25.984772 (+ 31us) server_negotiation.cc:407] Connection header received
0926 14:56:25.984814 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:25.984814 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:25.984822 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:25.984883 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:25.985257 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.985674 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.986278 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.986467 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.986511 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.988712 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:25.988713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:25.988713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:25.988724 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:25.989997 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:25.990303 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:25.990303 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:25.990304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:25.990342 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:25.990470 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:25.990470 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:25.990470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:25.990550 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:25.990613 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:25.990680 (+ 67us) server_negotiation.cc:299] Negotiation successful
0926 14:56:25.990727 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 14:56:25.991953 14815 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:56:25.993342 14815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:56:25.998857 15319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.995032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:37885 (local address 127.0.0.1:43608)
0926 14:56:25.995221 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:25.995254 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:25.995324 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:25.995534 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:25.995536 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:25.995538 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:25.995591 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:25.995592 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.996837 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.996838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:25.997859 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:25.997861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.998596 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.998597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:25.998670 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.998683 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 14:56:25.998746 (+ 63us) client_negotiation.cc:240] Negotiation successful
0926 14:56:25.998792 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":55,"threads_started":1}
I20260926 14:56:25.999071 15318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.995032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:41361 (local address 127.0.0.1:54984)
0926 14:56:25.995382 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:25.995392 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:25.995446 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:25.995630 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:25.995630 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:25.995632 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:25.995679 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:25.995681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.996576 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.996577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:25.997879 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:25.997882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.998855 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.998856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:25.998924 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.998927 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:25.998969 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 14:56:25.999005 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":29,"thread_start_us":44,"threads_started":1}
I20260926 14:56:25.999087 15322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.995230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43608 (local address 127.7.37.194:37885)
0926 14:56:25.995403 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:25.995407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:25.995415 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:25.995447 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:25.995449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:25.995462 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:25.995523 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:25.995649 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.996699 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.997998 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.998534 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.998595 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.998662 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:25.998760 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 14:56:25.998802 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":85,"threads_started":1}
I20260926 14:56:25.999328 15320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.995078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54984 (local address 127.7.37.195:41361)
0926 14:56:25.995213 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:25.995217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:25.995438 (+ 221us) server_negotiation.cc:407] Connection header received
0926 14:56:25.995500 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:25.995502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:25.995515 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:25.995581 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:25.995771 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.996507 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.997967 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.998667 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.998707 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.998771 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:25.999218 (+ 447us) server_negotiation.cc:299] Negotiation successful
0926 14:56:25.999252 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
I20260926 14:56:25.999492 15317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.995038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:35495 (local address 127.0.0.1:52328)
0926 14:56:25.995512 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:25.995520 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:25.995559 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:25.995726 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:25.995727 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:25.995728 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:25.995783 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:25.995784 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.996977 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.996978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:25.998133 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:25.998135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.999279 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.999280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:25.999337 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.999341 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:56:25.999401 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 14:56:25.999436 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":42,"threads_started":1}
I20260926 14:56:25.999692 15321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:25.995232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52328 (local address 127.7.37.193:35495)
0926 14:56:25.995384 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:25.995388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:25.995607 (+ 219us) server_negotiation.cc:407] Connection header received
0926 14:56:25.995645 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:25.995647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:25.995661 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:25.995730 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:25.995917 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.996911 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.998231 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:25.999126 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:25.999185 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:25.999249 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:25.999577 (+ 328us) server_negotiation.cc:299] Negotiation successful
0926 14:56:25.999615 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260926 14:56:25.999621 15092 tablet_service.cc:1467] Processing CreateTablet for tablet bf01b08e3d6642c4889f6ffb89480305 (DEFAULT_TABLE table=test-table [id=ede165eb87dc42b0b7f09ac4d044cc71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:25.999971 15092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf01b08e3d6642c4889f6ffb89480305. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:25.999864 15232 tablet_service.cc:1467] Processing CreateTablet for tablet bf01b08e3d6642c4889f6ffb89480305 (DEFAULT_TABLE table=test-table [id=ede165eb87dc42b0b7f09ac4d044cc71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:26.000151 15232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf01b08e3d6642c4889f6ffb89480305. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:26.003245 15323 tablet_bootstrap.cc:492] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98: Bootstrap starting.
I20260926 14:56:26.003844 15323 tablet_bootstrap.cc:654] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:26.003846 14952 tablet_service.cc:1467] Processing CreateTablet for tablet bf01b08e3d6642c4889f6ffb89480305 (DEFAULT_TABLE table=test-table [id=ede165eb87dc42b0b7f09ac4d044cc71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:26.003962 15324 tablet_bootstrap.cc:492] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac: Bootstrap starting.
I20260926 14:56:26.004130 14952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf01b08e3d6642c4889f6ffb89480305. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:26.004145 15323 log.cc:824] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:26.004489 15324 tablet_bootstrap.cc:654] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:26.004774 15324 log.cc:824] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:26.004869 15323 tablet_bootstrap.cc:492] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98: No bootstrap required, opened a new log
I20260926 14:56:26.004928 15323 ts_tablet_manager.cc:1397] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 14:56:26.005513 15324 tablet_bootstrap.cc:492] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac: No bootstrap required, opened a new log
I20260926 14:56:26.005573 15324 ts_tablet_manager.cc:1397] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:56:26.006709 15323 raft_consensus.cc:348] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } } peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } }
I20260926 14:56:26.006863 15323 raft_consensus.cc:374] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:26.006897 15323 raft_consensus.cc:729] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ad97f8c2524129bbe02a47e291ee98, State: Initialized, Role: FOLLOWER
I20260926 14:56:26.007025 15323 consensus_queue.cc:260] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } } peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } }
I20260926 14:56:26.007254 15323 ts_tablet_manager.cc:1428] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260926 14:56:26.007243 15327 tablet_bootstrap.cc:492] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d: Bootstrap starting.
I20260926 14:56:26.007370 15298 heartbeater.cc:503] Master 127.7.37.254:36783 was elected leader, sending a full tablet report...
I20260926 14:56:26.007438 15324 raft_consensus.cc:348] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } } peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } }
I20260926 14:56:26.007568 15324 raft_consensus.cc:374] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:26.007597 15324 raft_consensus.cc:729] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e1f78bf01a6418da2dded31464d28ac, State: Initialized, Role: FOLLOWER
I20260926 14:56:26.007697 15324 consensus_queue.cc:260] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } } peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } }
I20260926 14:56:26.007877 15327 tablet_bootstrap.cc:654] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:26.007951 15324 ts_tablet_manager.cc:1428] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 14:56:26.007997 15158 heartbeater.cc:503] Master 127.7.37.254:36783 was elected leader, sending a full tablet report...
I20260926 14:56:26.008160 15327 log.cc:824] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:26.008846 15327 tablet_bootstrap.cc:492] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d: No bootstrap required, opened a new log
I20260926 14:56:26.008903 15327 ts_tablet_manager.cc:1397] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:56:26.010136 15327 raft_consensus.cc:348] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } } peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } }
I20260926 14:56:26.010246 15327 raft_consensus.cc:374] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:26.010265 15327 raft_consensus.cc:729] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df829259b2534be29a2257d356c85a9d, State: Initialized, Role: FOLLOWER
I20260926 14:56:26.010345 15327 consensus_queue.cc:260] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } } peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } }
I20260926 14:56:26.010589 15327 ts_tablet_manager.cc:1428] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:56:26.010648 15018 heartbeater.cc:503] Master 127.7.37.254:36783 was elected leader, sending a full tablet report...
W20260926 14:56:26.075089 15159 tablet.cc:2369] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:56:26.104192 15329 raft_consensus.cc:482] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:56:26.104277 15329 raft_consensus.cc:504] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } } peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } }
I20260926 14:56:26.104537 15329 leader_election.cc:288] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53ad97f8c2524129bbe02a47e291ee98 (127.7.37.195:41361), df829259b2534be29a2257d356c85a9d (127.7.37.193:35495)
I20260926 14:56:26.107069 15321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:26.104795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:40099 (local address 127.7.37.193:35495)
0926 14:56:26.104845 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:26.104847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:26.104853 (+ 6us) server_negotiation.cc:407] Connection header received
0926 14:56:26.104875 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:26.104876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:26.104885 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:26.104930 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:26.105059 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.105531 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.106436 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.106776 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.106834 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:26.106842 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:26.106976 (+ 134us) server_negotiation.cc:299] Negotiation successful
0926 14:56:26.107007 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 14:56:26.107080 15051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:26.104696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:35495 (local address 127.7.37.194:40099)
0926 14:56:26.104741 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:26.104753 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:26.104809 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:26.104941 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:26.104942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:26.104943 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:26.105004 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:26.105006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.105598 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.105599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:26.106357 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:26.106359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.106841 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.106842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:26.106918 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:26.106920 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:56:26.106962 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 14:56:26.106999 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260926 14:56:26.107180 15320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:26.104764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:33291 (local address 127.7.37.195:41361)
0926 14:56:26.104820 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:26.104821 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:26.104958 (+ 137us) server_negotiation.cc:407] Connection header received
0926 14:56:26.105002 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:26.105002 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:26.105012 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:26.105058 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:26.105188 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.105703 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.106549 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.106896 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.106953 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:26.106960 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:26.107092 (+ 132us) server_negotiation.cc:299] Negotiation successful
0926 14:56:26.107121 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 14:56:26.107196 15332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:26.104696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:41361 (local address 127.7.37.194:33291)
0926 14:56:26.104885 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:26.104894 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:26.104948 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:26.105064 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:26.105065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:26.105066 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:26.105133 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:26.105135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.105783 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.105784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:26.106472 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:26.106473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.106962 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.106963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:26.107034 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:26.107037 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:26.107080 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 14:56:26.107114 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":61,"thread_start_us":84,"threads_started":1}
I20260926 14:56:26.107378 14972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf01b08e3d6642c4889f6ffb89480305" candidate_uuid: "0e1f78bf01a6418da2dded31464d28ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df829259b2534be29a2257d356c85a9d" is_pre_election: true
I20260926 14:56:26.107518 14972 raft_consensus.cc:2454] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e1f78bf01a6418da2dded31464d28ac in term 0.
I20260926 14:56:26.107519 15252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf01b08e3d6642c4889f6ffb89480305" candidate_uuid: "0e1f78bf01a6418da2dded31464d28ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ad97f8c2524129bbe02a47e291ee98" is_pre_election: true
I20260926 14:56:26.107637 15252 raft_consensus.cc:2454] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e1f78bf01a6418da2dded31464d28ac in term 0.
I20260926 14:56:26.107784 15043 leader_election.cc:302] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e1f78bf01a6418da2dded31464d28ac, df829259b2534be29a2257d356c85a9d; no voters:
I20260926 14:56:26.107923 15329 raft_consensus.cc:2781] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:56:26.107978 15329 raft_consensus.cc:482] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:56:26.108001 15329 raft_consensus.cc:3037] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:26.109025 15329 raft_consensus.cc:504] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } } peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } }
I20260926 14:56:26.109126 15329 leader_election.cc:288] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [CANDIDATE]: Term 1 election: Requested vote from peers 53ad97f8c2524129bbe02a47e291ee98 (127.7.37.195:41361), df829259b2534be29a2257d356c85a9d (127.7.37.193:35495)
I20260926 14:56:26.109282 14972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf01b08e3d6642c4889f6ffb89480305" candidate_uuid: "0e1f78bf01a6418da2dded31464d28ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df829259b2534be29a2257d356c85a9d"
I20260926 14:56:26.109287 15252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf01b08e3d6642c4889f6ffb89480305" candidate_uuid: "0e1f78bf01a6418da2dded31464d28ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ad97f8c2524129bbe02a47e291ee98"
I20260926 14:56:26.109349 14972 raft_consensus.cc:3037] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:26.109355 15252 raft_consensus.cc:3037] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:26.109943 14972 raft_consensus.cc:2454] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e1f78bf01a6418da2dded31464d28ac in term 1.
I20260926 14:56:26.110026 15252 raft_consensus.cc:2454] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e1f78bf01a6418da2dded31464d28ac in term 1.
I20260926 14:56:26.110080 15043 leader_election.cc:302] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e1f78bf01a6418da2dded31464d28ac, df829259b2534be29a2257d356c85a9d; no voters:
I20260926 14:56:26.110155 15329 raft_consensus.cc:2781] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:26.110282 15329 raft_consensus.cc:686] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [term 1 LEADER]: Becoming Leader. State: Replica: 0e1f78bf01a6418da2dded31464d28ac, State: Running, Role: LEADER
I20260926 14:56:26.110355 15329 consensus_queue.cc:237] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } } peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } }
I20260926 14:56:26.111018 14814 catalog_manager.cc:5495] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e1f78bf01a6418da2dded31464d28ac (127.7.37.194). New cstate: current_term: 1 leader_uuid: "0e1f78bf01a6418da2dded31464d28ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } health_report { overall_health: UNKNOWN } } }
I20260926 14:56:26.141392 15322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:26.139293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43610 (local address 127.7.37.194:37885)
0926 14:56:26.139351 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:26.139354 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:26.139379 (+ 25us) server_negotiation.cc:407] Connection header received
0926 14:56:26.139431 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:26.139432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:26.139449 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:26.139505 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:26.139603 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.140113 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.140790 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.140972 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.141028 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:26.141132 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:26.141225 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:26.141285 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:26.141304 (+ 19us) server_negotiation.cc:299] Negotiation successful
0926 14:56:26.141320 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 14:56:26.142979 15252 raft_consensus.cc:1261] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98 [term 1 FOLLOWER]: Refusing update from remote peer 0e1f78bf01a6418da2dded31464d28ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:26.142979 14972 raft_consensus.cc:1261] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d [term 1 FOLLOWER]: Refusing update from remote peer 0e1f78bf01a6418da2dded31464d28ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:26.143337 15333 consensus_queue.cc:1035] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:26.143472 15329 consensus_queue.cc:1035] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:26.146255 15338 mvcc.cc:205] Tried to move back new op lower bound from 7333620064839127040 to 7333620064708493312. Current Snapshot: MvccSnapshot[applied={T|T < 7333620064839127040}]
I20260926 14:56:26.146508 15336 mvcc.cc:205] Tried to move back new op lower bound from 7333620064839127040 to 7333620064708493312. Current Snapshot: MvccSnapshot[applied={T|T < 7333620064839127040 or (T in {7333620064839127040})}]
I20260926 14:56:26.146761 15337 mvcc.cc:205] Tried to move back new op lower bound from 7333620064839127040 to 7333620064708493312. Current Snapshot: MvccSnapshot[applied={T|T < 7333620064839127040}]
I20260926 14:56:26.147910 14815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:26.147989 14815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:26.148691 14815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a: Sending DeleteTablet for 3 replicas of tablet bf01b08e3d6642c4889f6ffb89480305
I20260926 14:56:26.148974 15092 tablet_service.cc:1514] Processing DeleteTablet for tablet bf01b08e3d6642c4889f6ffb89480305 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:26 UTC) from {username='oryx', principal='oryx/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:43608
I20260926 14:56:26.149106 14952 tablet_service.cc:1514] Processing DeleteTablet for tablet bf01b08e3d6642c4889f6ffb89480305 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:26 UTC) from {username='oryx', principal='oryx/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:52328
I20260926 14:56:26.149168 15232 tablet_service.cc:1514] Processing DeleteTablet for tablet bf01b08e3d6642c4889f6ffb89480305 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:26 UTC) from {username='oryx', principal='oryx/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:54984
I20260926 14:56:26.149283 14815 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 14:56:26.149369 15347 tablet_replica.cc:330] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98: stopping tablet replica
I20260926 14:56:26.149396 15345 tablet_replica.cc:330] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac: stopping tablet replica
W20260926 14:56:26.149407 14815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:56:26.149456 15347 raft_consensus.cc:2229] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:26.149482 15345 raft_consensus.cc:2229] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:56:26.149559 15345 raft_consensus.cc:2258] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:26.149773 15347 raft_consensus.cc:2258] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:26.149960 15345 ts_tablet_manager.cc:1905] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:26.150170 15347 ts_tablet_manager.cc:1905] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:26.150656 14952 tablet_service.cc:1467] Processing CreateTablet for tablet 79170e3d134843908a2313db099a0937 (DEFAULT_TABLE table=test-table [id=27d93d7995b74086a4c7dbece92b504d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:26.150763 14952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79170e3d134843908a2313db099a0937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:26.151479 15345 ts_tablet_manager.cc:1918] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:56:26.151532 15345 log.cc:1196] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/bf01b08e3d6642c4889f6ffb89480305
I20260926 14:56:26.151683 15347 ts_tablet_manager.cc:1918] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:56:26.151775 15347 log.cc:1196] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/bf01b08e3d6642c4889f6ffb89480305
I20260926 14:56:26.151775 15345 ts_tablet_manager.cc:1939] T bf01b08e3d6642c4889f6ffb89480305 P 0e1f78bf01a6418da2dded31464d28ac: Deleting consensus metadata
I20260926 14:56:26.152055 15347 ts_tablet_manager.cc:1939] T bf01b08e3d6642c4889f6ffb89480305 P 53ad97f8c2524129bbe02a47e291ee98: Deleting consensus metadata
I20260926 14:56:26.152314 15327 tablet_bootstrap.cc:492] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d: Bootstrap starting.
I20260926 14:56:26.152457 14802 catalog_manager.cc:4841] TS 0e1f78bf01a6418da2dded31464d28ac (127.7.37.194:37885): tablet bf01b08e3d6642c4889f6ffb89480305 (table test-table [id=ede165eb87dc42b0b7f09ac4d044cc71]) successfully deleted
I20260926 14:56:26.152640 15092 tablet_service.cc:1467] Processing CreateTablet for tablet 79170e3d134843908a2313db099a0937 (DEFAULT_TABLE table=test-table [id=27d93d7995b74086a4c7dbece92b504d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:26.152719 15092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79170e3d134843908a2313db099a0937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:26.152714 15327 tablet_bootstrap.cc:654] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:26.153311 15327 tablet_bootstrap.cc:492] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d: No bootstrap required, opened a new log
I20260926 14:56:26.153358 15327 ts_tablet_manager.cc:1397] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:26.153493 15327 raft_consensus.cc:348] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.153553 15327 raft_consensus.cc:374] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:26.153573 15327 raft_consensus.cc:729] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df829259b2534be29a2257d356c85a9d, State: Initialized, Role: FOLLOWER
I20260926 14:56:26.153616 15327 consensus_queue.cc:260] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.153694 15327 ts_tablet_manager.cc:1428] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:26.153733 15324 tablet_bootstrap.cc:492] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac: Bootstrap starting.
I20260926 14:56:26.153931 14800 catalog_manager.cc:4841] TS 53ad97f8c2524129bbe02a47e291ee98 (127.7.37.195:41361): tablet bf01b08e3d6642c4889f6ffb89480305 (table test-table [id=ede165eb87dc42b0b7f09ac4d044cc71]) successfully deleted
I20260926 14:56:26.154119 15232 tablet_service.cc:1467] Processing CreateTablet for tablet 79170e3d134843908a2313db099a0937 (DEFAULT_TABLE table=test-table [id=27d93d7995b74086a4c7dbece92b504d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:26.154171 15324 tablet_bootstrap.cc:654] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:26.154203 15232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79170e3d134843908a2313db099a0937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:26.154634 15324 tablet_bootstrap.cc:492] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac: No bootstrap required, opened a new log
I20260926 14:56:26.154676 15324 ts_tablet_manager.cc:1397] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:26.154803 15324 raft_consensus.cc:348] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.154874 15324 raft_consensus.cc:374] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:26.154892 15324 raft_consensus.cc:729] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e1f78bf01a6418da2dded31464d28ac, State: Initialized, Role: FOLLOWER
I20260926 14:56:26.154927 15324 consensus_queue.cc:260] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.155001 15324 ts_tablet_manager.cc:1428] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:26.155233 15323 tablet_bootstrap.cc:492] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98: Bootstrap starting.
I20260926 14:56:26.155613 15323 tablet_bootstrap.cc:654] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:26.155846 15346 tablet_replica.cc:330] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d: stopping tablet replica
I20260926 14:56:26.155934 15346 raft_consensus.cc:2229] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:26.156028 15346 raft_consensus.cc:2258] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:26.156189 15323 tablet_bootstrap.cc:492] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98: No bootstrap required, opened a new log
I20260926 14:56:26.156229 15323 ts_tablet_manager.cc:1397] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 14:56:26.156354 15323 raft_consensus.cc:348] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.156406 15323 raft_consensus.cc:374] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:26.156417 15323 raft_consensus.cc:729] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ad97f8c2524129bbe02a47e291ee98, State: Initialized, Role: FOLLOWER
I20260926 14:56:26.156446 15323 consensus_queue.cc:260] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.156519 15323 ts_tablet_manager.cc:1428] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:26.156785 15346 ts_tablet_manager.cc:1905] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:26.157982 15346 ts_tablet_manager.cc:1918] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:56:26.158032 15346 log.cc:1196] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/wal/wals/bf01b08e3d6642c4889f6ffb89480305
I20260926 14:56:26.158171 15346 ts_tablet_manager.cc:1939] T bf01b08e3d6642c4889f6ffb89480305 P df829259b2534be29a2257d356c85a9d: Deleting consensus metadata
I20260926 14:56:26.158493 14799 catalog_manager.cc:4841] TS df829259b2534be29a2257d356c85a9d (127.7.37.193:35495): tablet bf01b08e3d6642c4889f6ffb89480305 (table test-table [id=ede165eb87dc42b0b7f09ac4d044cc71]) successfully deleted
W20260926 14:56:26.177201 15019 tablet.cc:2369] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:56:26.180923 15331 raft_consensus.cc:482] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:56:26.180985 15331 raft_consensus.cc:504] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.181236 15331 leader_election.cc:288] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e1f78bf01a6418da2dded31464d28ac (127.7.37.194:37885), 53ad97f8c2524129bbe02a47e291ee98 (127.7.37.195:41361)
I20260926 14:56:26.184652 15320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:26.181454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:57021 (local address 127.7.37.195:41361)
0926 14:56:26.181490 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:26.181492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:26.181604 (+ 112us) server_negotiation.cc:407] Connection header received
0926 14:56:26.181700 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:26.181701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:26.181709 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:26.181768 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:26.181908 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.182363 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.183768 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.184349 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.184420 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:26.184429 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:26.184559 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 14:56:26.184597 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 14:56:26.184659 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:26.181361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:41361 (local address 127.7.37.193:57021)
0926 14:56:26.181544 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:26.181554 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:26.181625 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:26.181804 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:26.181805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:26.181806 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:26.181850 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:26.181851 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.182429 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.182430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:26.183667 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:26.183670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.184418 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.184419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:26.184491 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:26.184494 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:26.184543 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 14:56:26.184585 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":1,"thread_start_us":52,"threads_started":1}
I20260926 14:56:26.185206 15252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79170e3d134843908a2313db099a0937" candidate_uuid: "df829259b2534be29a2257d356c85a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ad97f8c2524129bbe02a47e291ee98" is_pre_election: true
I20260926 14:56:26.185314 15252 raft_consensus.cc:2454] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df829259b2534be29a2257d356c85a9d in term 0.
I20260926 14:56:26.185442 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:26.181361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:37885 (local address 127.7.37.193:47457)
0926 14:56:26.181459 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:26.181469 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:26.181514 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:26.181648 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:26.181649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:26.181651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:26.181707 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:26.181709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.182580 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.182581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:26.183668 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:26.183670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.185195 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.185196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:26.185265 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:26.185269 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:56:26.185324 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 14:56:26.185360 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":2,"thread_start_us":43,"threads_started":1}
I20260926 14:56:26.185457 14904 leader_election.cc:302] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ad97f8c2524129bbe02a47e291ee98, df829259b2534be29a2257d356c85a9d; no voters:
I20260926 14:56:26.185503 15322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:26.181463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:47457 (local address 127.7.37.194:37885)
0926 14:56:26.181499 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:26.181501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:26.181509 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:26.181591 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:26.181591 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:26.181600 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:26.181645 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:26.181773 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.182505 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.184474 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.185001 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.185078 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:26.185085 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:26.185408 (+ 323us) server_negotiation.cc:299] Negotiation successful
0926 14:56:26.185442 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 14:56:26.185546 15331 raft_consensus.cc:2781] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:56:26.185572 15331 raft_consensus.cc:482] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:56:26.185588 15331 raft_consensus.cc:3037] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:26.185699 15112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79170e3d134843908a2313db099a0937" candidate_uuid: "df829259b2534be29a2257d356c85a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1f78bf01a6418da2dded31464d28ac" is_pre_election: true
I20260926 14:56:26.185806 15112 raft_consensus.cc:2454] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df829259b2534be29a2257d356c85a9d in term 0.
I20260926 14:56:26.186240 15331 raft_consensus.cc:504] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.186340 15331 leader_election.cc:288] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [CANDIDATE]: Term 1 election: Requested vote from peers 0e1f78bf01a6418da2dded31464d28ac (127.7.37.194:37885), 53ad97f8c2524129bbe02a47e291ee98 (127.7.37.195:41361)
I20260926 14:56:26.186461 15112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79170e3d134843908a2313db099a0937" candidate_uuid: "df829259b2534be29a2257d356c85a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1f78bf01a6418da2dded31464d28ac"
I20260926 14:56:26.186525 15112 raft_consensus.cc:3037] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:26.186522 15252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79170e3d134843908a2313db099a0937" candidate_uuid: "df829259b2534be29a2257d356c85a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ad97f8c2524129bbe02a47e291ee98"
I20260926 14:56:26.186575 15252 raft_consensus.cc:3037] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:26.187158 15252 raft_consensus.cc:2454] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df829259b2534be29a2257d356c85a9d in term 1.
I20260926 14:56:26.187204 15112 raft_consensus.cc:2454] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df829259b2534be29a2257d356c85a9d in term 1.
I20260926 14:56:26.187311 14904 leader_election.cc:302] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ad97f8c2524129bbe02a47e291ee98, df829259b2534be29a2257d356c85a9d; no voters:
I20260926 14:56:26.187415 15331 raft_consensus.cc:2781] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:26.187513 15331 raft_consensus.cc:686] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [term 1 LEADER]: Becoming Leader. State: Replica: df829259b2534be29a2257d356c85a9d, State: Running, Role: LEADER
I20260926 14:56:26.187572 15331 consensus_queue.cc:237] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.188071 14814 catalog_manager.cc:5495] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d reported cstate change: term changed from 0 to 1, leader changed from <none> to df829259b2534be29a2257d356c85a9d (127.7.37.193). New cstate: current_term: 1 leader_uuid: "df829259b2534be29a2257d356c85a9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } health_report { overall_health: UNKNOWN } } }
W20260926 14:56:26.209432 15299 tablet.cc:2369] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 14:56:26 dist-test-slave-f9xb krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434585, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.254@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM
I20260926 14:56:26.215044 15360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:26.208265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36783 (local address 127.0.0.1:54264)
0926 14:56:26.208391 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:26.208401 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:26.208451 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:26.208555 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:26.208556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:26.208813 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:26.208850 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:26.208852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.209730 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.209731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:26.210229 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:26.210230 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.210694 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.210694 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:26.210741 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:26.211324 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:26.211328 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:26.212592 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:26.214165 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:26.214171 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:26.214172 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:26.214351 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:26.214541 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:26.214541 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:26.214542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:26.214589 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:26.214823 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:26.214826 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:26.214862 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 14:56:26.214905 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 14:56:26.214981 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
I20260926 14:56:26.215131 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:26.208339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54264 (local address 127.7.37.254:36783)
0926 14:56:26.208384 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:26.208386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:26.208459 (+ 73us) server_negotiation.cc:407] Connection header received
0926 14:56:26.208482 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:26.208482 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:26.208492 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:26.208553 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:26.208918 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.209654 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.210320 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.210595 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.210668 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:26.212677 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:26.212678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:26.212679 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:26.212685 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:26.214093 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:26.214418 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:26.214419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:26.214419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:26.214480 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:26.214640 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:26.214641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:26.214641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:26.214766 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:26.214835 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:26.214933 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 14:56:26.215048 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":26}
I20260926 14:56:26.216882 14814 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:54264:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 14:56:26.217078 14814 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:56:26.218135 14952 tablet_service.cc:1467] Processing CreateTablet for tablet 288017df7bc947b09b2294f2d9c8e7ae (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dbc0bce7837f44b691b053cb76a1724a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 14:56:26.218149 15232 tablet_service.cc:1467] Processing CreateTablet for tablet 288017df7bc947b09b2294f2d9c8e7ae (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dbc0bce7837f44b691b053cb76a1724a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 14:56:26.218261 15232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 288017df7bc947b09b2294f2d9c8e7ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:26.218261 14952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 288017df7bc947b09b2294f2d9c8e7ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:26.219542 15323 tablet_bootstrap.cc:492] T 288017df7bc947b09b2294f2d9c8e7ae P 53ad97f8c2524129bbe02a47e291ee98: Bootstrap starting.
I20260926 14:56:26.219640 15092 tablet_service.cc:1467] Processing CreateTablet for tablet 288017df7bc947b09b2294f2d9c8e7ae (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dbc0bce7837f44b691b053cb76a1724a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 14:56:26.219736 15092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 288017df7bc947b09b2294f2d9c8e7ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:26.219805 15327 tablet_bootstrap.cc:492] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d: Bootstrap starting.
I20260926 14:56:26.220165 15327 tablet_bootstrap.cc:654] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:26.220645 15323 tablet_bootstrap.cc:654] T 288017df7bc947b09b2294f2d9c8e7ae P 53ad97f8c2524129bbe02a47e291ee98: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:26.220667 15324 tablet_bootstrap.cc:492] T 288017df7bc947b09b2294f2d9c8e7ae P 0e1f78bf01a6418da2dded31464d28ac: Bootstrap starting.
I20260926 14:56:26.220728 15327 tablet_bootstrap.cc:492] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d: No bootstrap required, opened a new log
I20260926 14:56:26.220763 15327 ts_tablet_manager.cc:1397] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:26.220903 15327 raft_consensus.cc:348] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.220969 15327 raft_consensus.cc:374] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:26.220988 15327 raft_consensus.cc:729] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df829259b2534be29a2257d356c85a9d, State: Initialized, Role: FOLLOWER
I20260926 14:56:26.221017 15327 consensus_queue.cc:260] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.221098 15327 ts_tablet_manager.cc:1428] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:26.221212 15324 tablet_bootstrap.cc:654] T 288017df7bc947b09b2294f2d9c8e7ae P 0e1f78bf01a6418da2dded31464d28ac: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:26.221544 15331 tablet_replica.cc:439] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } } }
I20260926 14:56:26.221613 15331 tablet_replica.cc:442] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:26.221777 15324 tablet_bootstrap.cc:492] T 288017df7bc947b09b2294f2d9c8e7ae P 0e1f78bf01a6418da2dded31464d28ac: No bootstrap required, opened a new log
I20260926 14:56:26.221825 15324 ts_tablet_manager.cc:1397] T 288017df7bc947b09b2294f2d9c8e7ae P 0e1f78bf01a6418da2dded31464d28ac: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:26.221948 15324 raft_consensus.cc:348] T 288017df7bc947b09b2294f2d9c8e7ae P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.222009 15324 raft_consensus.cc:374] T 288017df7bc947b09b2294f2d9c8e7ae P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:26.222026 15324 raft_consensus.cc:729] T 288017df7bc947b09b2294f2d9c8e7ae P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e1f78bf01a6418da2dded31464d28ac, State: Initialized, Role: FOLLOWER
I20260926 14:56:26.222066 15324 consensus_queue.cc:260] T 288017df7bc947b09b2294f2d9c8e7ae P 0e1f78bf01a6418da2dded31464d28ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.222144 15324 ts_tablet_manager.cc:1428] T 288017df7bc947b09b2294f2d9c8e7ae P 0e1f78bf01a6418da2dded31464d28ac: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:26.222193 15323 tablet_bootstrap.cc:492] T 288017df7bc947b09b2294f2d9c8e7ae P 53ad97f8c2524129bbe02a47e291ee98: No bootstrap required, opened a new log
I20260926 14:56:26.222231 15323 ts_tablet_manager.cc:1397] T 288017df7bc947b09b2294f2d9c8e7ae P 53ad97f8c2524129bbe02a47e291ee98: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260926 14:56:26.222216 15333 tablet_replica.cc:439] T 288017df7bc947b09b2294f2d9c8e7ae P 0e1f78bf01a6418da2dded31464d28ac: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } } }
I20260926 14:56:26.222271 15333 tablet_replica.cc:442] T 288017df7bc947b09b2294f2d9c8e7ae P 0e1f78bf01a6418da2dded31464d28ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:26.222364 15323 raft_consensus.cc:348] T 288017df7bc947b09b2294f2d9c8e7ae P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.222433 15323 raft_consensus.cc:374] T 288017df7bc947b09b2294f2d9c8e7ae P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:26.222452 15323 raft_consensus.cc:729] T 288017df7bc947b09b2294f2d9c8e7ae P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ad97f8c2524129bbe02a47e291ee98, State: Initialized, Role: FOLLOWER
I20260926 14:56:26.222491 15323 consensus_queue.cc:260] T 288017df7bc947b09b2294f2d9c8e7ae P 53ad97f8c2524129bbe02a47e291ee98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.222572 15323 ts_tablet_manager.cc:1428] T 288017df7bc947b09b2294f2d9c8e7ae P 53ad97f8c2524129bbe02a47e291ee98: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:26.222692 15328 tablet_replica.cc:439] T 288017df7bc947b09b2294f2d9c8e7ae P 53ad97f8c2524129bbe02a47e291ee98: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } } }
I20260926 14:56:26.222746 15328 tablet_replica.cc:442] T 288017df7bc947b09b2294f2d9c8e7ae P 53ad97f8c2524129bbe02a47e291ee98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:26.346225 15331 raft_consensus.cc:482] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:56:26.346305 15331 raft_consensus.cc:504] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.346478 15331 leader_election.cc:288] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e1f78bf01a6418da2dded31464d28ac (127.7.37.194:37885), 53ad97f8c2524129bbe02a47e291ee98 (127.7.37.195:41361)
I20260926 14:56:26.346637 15112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "288017df7bc947b09b2294f2d9c8e7ae" candidate_uuid: "df829259b2534be29a2257d356c85a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1f78bf01a6418da2dded31464d28ac" is_pre_election: true
I20260926 14:56:26.346668 15252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "288017df7bc947b09b2294f2d9c8e7ae" candidate_uuid: "df829259b2534be29a2257d356c85a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ad97f8c2524129bbe02a47e291ee98" is_pre_election: true
I20260926 14:56:26.346738 15252 raft_consensus.cc:2454] T 288017df7bc947b09b2294f2d9c8e7ae P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df829259b2534be29a2257d356c85a9d in term 0.
I20260926 14:56:26.346738 15112 raft_consensus.cc:2454] T 288017df7bc947b09b2294f2d9c8e7ae P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df829259b2534be29a2257d356c85a9d in term 0.
I20260926 14:56:26.346889 14904 leader_election.cc:302] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0e1f78bf01a6418da2dded31464d28ac, 53ad97f8c2524129bbe02a47e291ee98, df829259b2534be29a2257d356c85a9d; no voters:
I20260926 14:56:26.346995 15331 raft_consensus.cc:2781] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:56:26.347038 15331 raft_consensus.cc:482] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:56:26.347054 15331 raft_consensus.cc:3037] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:26.347708 15331 raft_consensus.cc:504] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.347868 15331 leader_election.cc:288] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d [CANDIDATE]: Term 1 election: Requested vote from peers 0e1f78bf01a6418da2dded31464d28ac (127.7.37.194:37885), 53ad97f8c2524129bbe02a47e291ee98 (127.7.37.195:41361)
I20260926 14:56:26.347999 15252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "288017df7bc947b09b2294f2d9c8e7ae" candidate_uuid: "df829259b2534be29a2257d356c85a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ad97f8c2524129bbe02a47e291ee98"
I20260926 14:56:26.348063 15252 raft_consensus.cc:3037] T 288017df7bc947b09b2294f2d9c8e7ae P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:26.348018 15112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "288017df7bc947b09b2294f2d9c8e7ae" candidate_uuid: "df829259b2534be29a2257d356c85a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1f78bf01a6418da2dded31464d28ac"
I20260926 14:56:26.348089 15112 raft_consensus.cc:3037] T 288017df7bc947b09b2294f2d9c8e7ae P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:26.348575 15252 raft_consensus.cc:2454] T 288017df7bc947b09b2294f2d9c8e7ae P 53ad97f8c2524129bbe02a47e291ee98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df829259b2534be29a2257d356c85a9d in term 1.
I20260926 14:56:26.348726 14904 leader_election.cc:302] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ad97f8c2524129bbe02a47e291ee98, df829259b2534be29a2257d356c85a9d; no voters:
I20260926 14:56:26.348836 15112 raft_consensus.cc:2454] T 288017df7bc947b09b2294f2d9c8e7ae P 0e1f78bf01a6418da2dded31464d28ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df829259b2534be29a2257d356c85a9d in term 1.
I20260926 14:56:26.348836 15331 raft_consensus.cc:2781] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:26.348892 15331 raft_consensus.cc:686] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d [term 1 LEADER]: Becoming Leader. State: Replica: df829259b2534be29a2257d356c85a9d, State: Running, Role: LEADER
I20260926 14:56:26.348940 15331 consensus_queue.cc:237] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.349182 15353 tablet_replica.cc:439] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d: TxnStatusTablet state changed. Reason: New leader df829259b2534be29a2257d356c85a9d. Latest consensus state: current_term: 1 leader_uuid: "df829259b2534be29a2257d356c85a9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } } }
I20260926 14:56:26.349248 15353 tablet_replica.cc:442] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d: This TxnStatusTablet replica's current role is: LEADER
I20260926 14:56:26.349457 15365 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 14:56:26.349589 15365 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 14:56:26.349572 14814 catalog_manager.cc:5495] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d reported cstate change: term changed from 0 to 1, leader changed from <none> to df829259b2534be29a2257d356c85a9d (127.7.37.193). New cstate: current_term: 1 leader_uuid: "df829259b2534be29a2257d356c85a9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } health_report { overall_health: UNKNOWN } } }
I20260926 14:56:26.367354 15360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:26.364851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:35495 (local address 127.0.0.1:52330)
0926 14:56:26.364926 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:26.364936 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:26.364970 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:26.365113 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:26.365114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:26.365115 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 14:56:26.365154 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:26.365156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.366003 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.366003 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:26.366542 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:26.366544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.366955 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.366955 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:26.366994 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:26.367005 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 14:56:26.367235 (+ 230us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 14:56:26.367238 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:26.367264 (+ 26us) client_negotiation.cc:240] Negotiation successful
0926 14:56:26.367298 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 14:56:26.367369 15321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:26.364911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52330 (local address 127.7.37.193:35495)
0926 14:56:26.364951 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:26.364953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:26.364962 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:56:26.365026 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:26.365026 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:26.365043 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:26.365113 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:26.365191 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.365937 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.366601 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.366898 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.366959 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:26.367072 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:26.367182 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:26.367247 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:26.367267 (+ 20us) server_negotiation.cc:299] Negotiation successful
0926 14:56:26.367286 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 14:56:26.368240 15252 raft_consensus.cc:1261] T 288017df7bc947b09b2294f2d9c8e7ae P 53ad97f8c2524129bbe02a47e291ee98 [term 1 FOLLOWER]: Refusing update from remote peer df829259b2534be29a2257d356c85a9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:26.368371 15112 raft_consensus.cc:1261] T 288017df7bc947b09b2294f2d9c8e7ae P 0e1f78bf01a6418da2dded31464d28ac [term 1 FOLLOWER]: Refusing update from remote peer df829259b2534be29a2257d356c85a9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:26.368446 15353 consensus_queue.cc:1035] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:26.368952 15328 tablet_replica.cc:439] T 288017df7bc947b09b2294f2d9c8e7ae P 53ad97f8c2524129bbe02a47e291ee98: TxnStatusTablet state changed. Reason: New leader df829259b2534be29a2257d356c85a9d. Latest consensus state: current_term: 1 leader_uuid: "df829259b2534be29a2257d356c85a9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } } }
I20260926 14:56:26.369045 15328 tablet_replica.cc:442] T 288017df7bc947b09b2294f2d9c8e7ae P 53ad97f8c2524129bbe02a47e291ee98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:26.369120 15353 consensus_queue.cc:1035] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:26.369563 15331 tablet_replica.cc:439] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df829259b2534be29a2257d356c85a9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } } }
I20260926 14:56:26.369637 15331 tablet_replica.cc:442] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d: This TxnStatusTablet replica's current role is: LEADER
I20260926 14:56:26.369774 15333 tablet_replica.cc:439] T 288017df7bc947b09b2294f2d9c8e7ae P 0e1f78bf01a6418da2dded31464d28ac: TxnStatusTablet state changed. Reason: New leader df829259b2534be29a2257d356c85a9d. Latest consensus state: current_term: 1 leader_uuid: "df829259b2534be29a2257d356c85a9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } } }
I20260926 14:56:26.369872 15333 tablet_replica.cc:442] T 288017df7bc947b09b2294f2d9c8e7ae P 0e1f78bf01a6418da2dded31464d28ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:26.370077 15328 tablet_replica.cc:439] T 288017df7bc947b09b2294f2d9c8e7ae P 53ad97f8c2524129bbe02a47e291ee98: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df829259b2534be29a2257d356c85a9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } } }
I20260926 14:56:26.370157 15328 tablet_replica.cc:442] T 288017df7bc947b09b2294f2d9c8e7ae P 53ad97f8c2524129bbe02a47e291ee98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:26.370841 15353 tablet_replica.cc:439] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df829259b2534be29a2257d356c85a9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } } }
I20260926 14:56:26.370924 15353 tablet_replica.cc:442] T 288017df7bc947b09b2294f2d9c8e7ae P df829259b2534be29a2257d356c85a9d: This TxnStatusTablet replica's current role is: LEADER
I20260926 14:56:26.371258 15333 tablet_replica.cc:439] T 288017df7bc947b09b2294f2d9c8e7ae P 0e1f78bf01a6418da2dded31464d28ac: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df829259b2534be29a2257d356c85a9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } } }
I20260926 14:56:26.371332 15333 tablet_replica.cc:442] T 288017df7bc947b09b2294f2d9c8e7ae P 0e1f78bf01a6418da2dded31464d28ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 14:56:26.373750 15321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:26.371724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52344 (local address 127.7.37.193:35495)
0926 14:56:26.371774 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:26.371776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:26.371798 (+ 22us) server_negotiation.cc:407] Connection header received
0926 14:56:26.371861 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:26.371862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:26.371870 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:26.371916 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:26.372033 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.372482 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.373203 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.373398 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.373449 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:26.373565 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:26.373594 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:26.373633 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:26.373682 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 14:56:26.373696 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 14:56:26.378226 15321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:26.375881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:47011 (local address 127.7.37.193:35495)
0926 14:56:26.375919 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:26.375920 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:26.376050 (+ 130us) server_negotiation.cc:407] Connection header received
0926 14:56:26.376101 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:26.376101 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:26.376108 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:26.376146 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:26.376257 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.376670 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.377698 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.377859 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.377897 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:26.378049 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:26.378070 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:26.378096 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:26.378156 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 14:56:26.378171 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":20}
I20260926 14:56:26.378314 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:26.375823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:35495 (local address 127.7.37.193:47011)
0926 14:56:26.375989 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:26.375998 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:26.376052 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:26.376150 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:26.376150 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:26.376151 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 14:56:26.376207 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:26.376208 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.376727 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.376728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:26.377632 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:26.377634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.377912 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.377913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:26.377973 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:26.377987 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 14:56:26.378108 (+ 121us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 14:56:26.378110 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:56:26.378155 (+ 45us) client_negotiation.cc:240] Negotiation successful
0926 14:56:26.378192 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260926 14:56:26.380272 15252 raft_consensus.cc:1261] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98 [term 1 FOLLOWER]: Refusing update from remote peer df829259b2534be29a2257d356c85a9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:26.380275 15112 raft_consensus.cc:1261] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac [term 1 FOLLOWER]: Refusing update from remote peer df829259b2534be29a2257d356c85a9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:26.380476 15353 consensus_queue.cc:1035] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:26.380671 15353 consensus_queue.cc:1035] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:26.393038 14814 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:26.393117 14814 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:26.393610 14814 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a: Sending DeleteTablet for 3 replicas of tablet 79170e3d134843908a2313db099a0937
I20260926 14:56:26.393829 15232 tablet_service.cc:1514] Processing DeleteTablet for tablet 79170e3d134843908a2313db099a0937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:26 UTC) from {username='oryx', principal='oryx/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:54984
I20260926 14:56:26.393862 15092 tablet_service.cc:1514] Processing DeleteTablet for tablet 79170e3d134843908a2313db099a0937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:26 UTC) from {username='oryx', principal='oryx/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:43608
I20260926 14:56:26.393915 15347 tablet_replica.cc:330] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98: stopping tablet replica
I20260926 14:56:26.393963 15345 tablet_replica.cc:330] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac: stopping tablet replica
I20260926 14:56:26.393970 15347 raft_consensus.cc:2229] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:26.394007 15347 raft_consensus.cc:2258] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:26.394023 15345 raft_consensus.cc:2229] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:26.394061 15345 raft_consensus.cc:2258] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:26.394235 15347 ts_tablet_manager.cc:1905] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:26.394325 14952 tablet_service.cc:1514] Processing DeleteTablet for tablet 79170e3d134843908a2313db099a0937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:26 UTC) from {username='oryx', principal='oryx/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:52328
I20260926 14:56:26.394399 15346 tablet_replica.cc:330] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d: stopping tablet replica
I20260926 14:56:26.394443 15346 raft_consensus.cc:2229] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:56:26.394510 15346 raft_consensus.cc:2258] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:26.395704 15347 ts_tablet_manager.cc:1918] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 14:56:26.395774 15347 log.cc:1196] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/79170e3d134843908a2313db099a0937
I20260926 14:56:26.395967 15347 ts_tablet_manager.cc:1939] T 79170e3d134843908a2313db099a0937 P 53ad97f8c2524129bbe02a47e291ee98: Deleting consensus metadata
I20260926 14:56:26.396211 15346 ts_tablet_manager.cc:1905] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:26.396361 14800 catalog_manager.cc:4841] TS 53ad97f8c2524129bbe02a47e291ee98 (127.7.37.195:41361): tablet 79170e3d134843908a2313db099a0937 (table test-table [id=27d93d7995b74086a4c7dbece92b504d]) successfully deleted
I20260926 14:56:26.396787 15345 ts_tablet_manager.cc:1905] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:26.397625 15346 ts_tablet_manager.cc:1918] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 14:56:26.397670 15346 log.cc:1196] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/wal/wals/79170e3d134843908a2313db099a0937
I20260926 14:56:26.397857 15346 ts_tablet_manager.cc:1939] T 79170e3d134843908a2313db099a0937 P df829259b2534be29a2257d356c85a9d: Deleting consensus metadata
I20260926 14:56:26.398258 15345 ts_tablet_manager.cc:1918] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 14:56:26.398309 15345 log.cc:1196] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/79170e3d134843908a2313db099a0937
I20260926 14:56:26.398300 14799 catalog_manager.cc:4841] TS df829259b2534be29a2257d356c85a9d (127.7.37.193:35495): tablet 79170e3d134843908a2313db099a0937 (table test-table [id=27d93d7995b74086a4c7dbece92b504d]) successfully deleted
I20260926 14:56:26.398471 15345 ts_tablet_manager.cc:1939] T 79170e3d134843908a2313db099a0937 P 0e1f78bf01a6418da2dded31464d28ac: Deleting consensus metadata
I20260926 14:56:26.398836 14802 catalog_manager.cc:4841] TS 0e1f78bf01a6418da2dded31464d28ac (127.7.37.194:37885): tablet 79170e3d134843908a2313db099a0937 (table test-table [id=27d93d7995b74086a4c7dbece92b504d]) successfully deleted
Sep 26 14:56:26 dist-test-slave-f9xb krb5kdc[14770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 14:56:26 dist-test-slave-f9xb krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:26 dist-test-slave-f9xb krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 14:56:26.415676 14864 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:26.410896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54272 (local address 127.7.37.254:36783)
0926 14:56:26.410944 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:26.410956 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:26.411059 (+ 103us) server_negotiation.cc:407] Connection header received
0926 14:56:26.411113 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:26.411114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:26.411124 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:26.411164 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:26.411652 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.412151 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.412819 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.413009 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.413059 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:26.415621 (+ 2562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54272: BlockingRecv error: recv got EOF from 127.0.0.1:54272 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Sep 26 14:56:26 dist-test-slave-f9xb krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM
I20260926 14:56:26.422842 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:26.416344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54288 (local address 127.7.37.254:36783)
0926 14:56:26.416410 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:26.416411 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:26.416423 (+ 12us) server_negotiation.cc:407] Connection header received
0926 14:56:26.416484 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:26.416485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:26.416494 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:26.416551 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:26.416920 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.417586 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.418202 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.418480 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.418548 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:26.420616 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:26.420618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:26.420618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:26.420623 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:26.421773 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:26.422187 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:26.422188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:26.422188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:26.422269 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:26.422452 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:26.422453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:26.422453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:26.422553 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:26.422618 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:26.422736 (+ 118us) server_negotiation.cc:299] Negotiation successful
0926 14:56:26.422789 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 14:56:26.424211 14814 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:56:26.424366 14814 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:56:26.425446 14952 tablet_service.cc:1467] Processing CreateTablet for tablet 3163c76cf4d6401aa70491e3fda16f68 (DEFAULT_TABLE table=test-table [id=fb5a01b4e45147eaa12b28cfac644567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:26.425511 15092 tablet_service.cc:1467] Processing CreateTablet for tablet 3163c76cf4d6401aa70491e3fda16f68 (DEFAULT_TABLE table=test-table [id=fb5a01b4e45147eaa12b28cfac644567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:26.425566 14952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3163c76cf4d6401aa70491e3fda16f68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:26.425623 15092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3163c76cf4d6401aa70491e3fda16f68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:26.425769 15232 tablet_service.cc:1467] Processing CreateTablet for tablet 3163c76cf4d6401aa70491e3fda16f68 (DEFAULT_TABLE table=test-table [id=fb5a01b4e45147eaa12b28cfac644567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:26.425854 15232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3163c76cf4d6401aa70491e3fda16f68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:26.426810 15327 tablet_bootstrap.cc:492] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d: Bootstrap starting.
I20260926 14:56:26.426824 15324 tablet_bootstrap.cc:492] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac: Bootstrap starting.
I20260926 14:56:26.427114 15324 tablet_bootstrap.cc:654] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:26.427192 15327 tablet_bootstrap.cc:654] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:26.427302 15323 tablet_bootstrap.cc:492] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98: Bootstrap starting.
I20260926 14:56:26.427665 15323 tablet_bootstrap.cc:654] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:26.427736 15327 tablet_bootstrap.cc:492] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d: No bootstrap required, opened a new log
I20260926 14:56:26.427802 15327 ts_tablet_manager.cc:1397] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:26.427953 15327 raft_consensus.cc:348] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.428020 15327 raft_consensus.cc:374] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:26.428040 15327 raft_consensus.cc:729] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df829259b2534be29a2257d356c85a9d, State: Initialized, Role: FOLLOWER
I20260926 14:56:26.428081 15327 consensus_queue.cc:260] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.428159 15327 ts_tablet_manager.cc:1428] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:26.428485 15324 tablet_bootstrap.cc:492] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac: No bootstrap required, opened a new log
I20260926 14:56:26.428525 15324 ts_tablet_manager.cc:1397] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260926 14:56:26.428617 15324 raft_consensus.cc:348] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.428676 15324 raft_consensus.cc:374] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:26.428689 15324 raft_consensus.cc:729] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e1f78bf01a6418da2dded31464d28ac, State: Initialized, Role: FOLLOWER
I20260926 14:56:26.428725 15324 consensus_queue.cc:260] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.428807 15324 ts_tablet_manager.cc:1428] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:26.429116 15323 tablet_bootstrap.cc:492] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98: No bootstrap required, opened a new log
I20260926 14:56:26.429162 15323 ts_tablet_manager.cc:1397] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 14:56:26.429297 15323 raft_consensus.cc:348] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.429354 15323 raft_consensus.cc:374] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:26.429373 15323 raft_consensus.cc:729] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ad97f8c2524129bbe02a47e291ee98, State: Initialized, Role: FOLLOWER
I20260926 14:56:26.429406 15323 consensus_queue.cc:260] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.429500 15323 ts_tablet_manager.cc:1428] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:26.557747 15331 raft_consensus.cc:482] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:56:26.557821 15331 raft_consensus.cc:504] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.557962 15331 leader_election.cc:288] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e1f78bf01a6418da2dded31464d28ac (127.7.37.194:37885), 53ad97f8c2524129bbe02a47e291ee98 (127.7.37.195:41361)
I20260926 14:56:26.558138 15112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3163c76cf4d6401aa70491e3fda16f68" candidate_uuid: "df829259b2534be29a2257d356c85a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1f78bf01a6418da2dded31464d28ac" is_pre_election: true
I20260926 14:56:26.558132 15252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3163c76cf4d6401aa70491e3fda16f68" candidate_uuid: "df829259b2534be29a2257d356c85a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ad97f8c2524129bbe02a47e291ee98" is_pre_election: true
I20260926 14:56:26.558221 15112 raft_consensus.cc:2454] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df829259b2534be29a2257d356c85a9d in term 0.
I20260926 14:56:26.558226 15252 raft_consensus.cc:2454] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df829259b2534be29a2257d356c85a9d in term 0.
I20260926 14:56:26.558372 14904 leader_election.cc:302] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ad97f8c2524129bbe02a47e291ee98, df829259b2534be29a2257d356c85a9d; no voters:
I20260926 14:56:26.558472 15331 raft_consensus.cc:2781] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:56:26.558516 15331 raft_consensus.cc:482] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:56:26.558535 15331 raft_consensus.cc:3037] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:26.559340 15331 raft_consensus.cc:504] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.559474 15331 leader_election.cc:288] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [CANDIDATE]: Term 1 election: Requested vote from peers 0e1f78bf01a6418da2dded31464d28ac (127.7.37.194:37885), 53ad97f8c2524129bbe02a47e291ee98 (127.7.37.195:41361)
I20260926 14:56:26.559594 15252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3163c76cf4d6401aa70491e3fda16f68" candidate_uuid: "df829259b2534be29a2257d356c85a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ad97f8c2524129bbe02a47e291ee98"
I20260926 14:56:26.559595 15112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3163c76cf4d6401aa70491e3fda16f68" candidate_uuid: "df829259b2534be29a2257d356c85a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1f78bf01a6418da2dded31464d28ac"
I20260926 14:56:26.559670 15252 raft_consensus.cc:3037] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:26.559670 15112 raft_consensus.cc:3037] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:26.560366 15252 raft_consensus.cc:2454] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df829259b2534be29a2257d356c85a9d in term 1.
I20260926 14:56:26.560371 15112 raft_consensus.cc:2454] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df829259b2534be29a2257d356c85a9d in term 1.
I20260926 14:56:26.560529 14906 leader_election.cc:302] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e1f78bf01a6418da2dded31464d28ac, df829259b2534be29a2257d356c85a9d; no voters:
I20260926 14:56:26.560642 15331 raft_consensus.cc:2781] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:26.560710 15331 raft_consensus.cc:686] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [term 1 LEADER]: Becoming Leader. State: Replica: df829259b2534be29a2257d356c85a9d, State: Running, Role: LEADER
I20260926 14:56:26.560775 15331 consensus_queue.cc:237] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.561245 14815 catalog_manager.cc:5495] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d reported cstate change: term changed from 0 to 1, leader changed from <none> to df829259b2534be29a2257d356c85a9d (127.7.37.193). New cstate: current_term: 1 leader_uuid: "df829259b2534be29a2257d356c85a9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } health_report { overall_health: UNKNOWN } } }
I20260926 14:56:26.574117 15321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:26.571867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52346 (local address 127.7.37.193:35495)
0926 14:56:26.571905 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:26.571907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:26.571937 (+ 30us) server_negotiation.cc:407] Connection header received
0926 14:56:26.572000 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:26.572000 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:26.572009 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:26.572047 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:26.572139 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.572595 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.573536 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:26.573711 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:26.573766 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:26.573952 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:26.573976 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:26.574006 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:26.574055 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 14:56:26.574067 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 14:56:26.574687 15252 raft_consensus.cc:1261] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98 [term 1 FOLLOWER]: Refusing update from remote peer df829259b2534be29a2257d356c85a9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:26.574704 15112 raft_consensus.cc:1261] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac [term 1 FOLLOWER]: Refusing update from remote peer df829259b2534be29a2257d356c85a9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:26.574884 15370 consensus_queue.cc:1035] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:26.574976 15331 consensus_queue.cc:1035] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:26.577015 14815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:26.577095 14815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:26.577587 14815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a: Sending DeleteTablet for 3 replicas of tablet 3163c76cf4d6401aa70491e3fda16f68
I20260926 14:56:26.577797 15092 tablet_service.cc:1514] Processing DeleteTablet for tablet 3163c76cf4d6401aa70491e3fda16f68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:26 UTC) from {username='oryx', principal='oryx/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:43608
I20260926 14:56:26.577849 14952 tablet_service.cc:1514] Processing DeleteTablet for tablet 3163c76cf4d6401aa70491e3fda16f68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:26 UTC) from {username='oryx', principal='oryx/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:52328
I20260926 14:56:26.577886 15232 tablet_service.cc:1514] Processing DeleteTablet for tablet 3163c76cf4d6401aa70491e3fda16f68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:26 UTC) from {username='oryx', principal='oryx/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:54984
I20260926 14:56:26.577940 15346 tablet_replica.cc:330] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d: stopping tablet replica
I20260926 14:56:26.577966 15345 tablet_replica.cc:330] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac: stopping tablet replica
I20260926 14:56:26.577988 15346 raft_consensus.cc:2229] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:56:26.578020 15345 raft_consensus.cc:2229] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:26.578034 15346 raft_consensus.cc:2258] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:26.578063 15345 raft_consensus.cc:2258] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:26.578125 15347 tablet_replica.cc:330] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98: stopping tablet replica
I20260926 14:56:26.578169 15347 raft_consensus.cc:2229] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:26.578212 15347 raft_consensus.cc:2258] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:26.578343 15346 ts_tablet_manager.cc:1905] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:26.578504 15347 ts_tablet_manager.cc:1905] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:26.579401 14815 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:56:26.579528 14815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:56:26.579623 15345 ts_tablet_manager.cc:1905] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:26.579831 15347 ts_tablet_manager.cc:1918] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:56:26.579831 15346 ts_tablet_manager.cc:1918] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:56:26.579886 15346 log.cc:1196] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-0/wal/wals/3163c76cf4d6401aa70491e3fda16f68
I20260926 14:56:26.579895 15347 log.cc:1196] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/3163c76cf4d6401aa70491e3fda16f68
I20260926 14:56:26.580080 15346 ts_tablet_manager.cc:1939] T 3163c76cf4d6401aa70491e3fda16f68 P df829259b2534be29a2257d356c85a9d: Deleting consensus metadata
I20260926 14:56:26.580087 15347 ts_tablet_manager.cc:1939] T 3163c76cf4d6401aa70491e3fda16f68 P 53ad97f8c2524129bbe02a47e291ee98: Deleting consensus metadata
I20260926 14:56:26.580616 14799 catalog_manager.cc:4841] TS df829259b2534be29a2257d356c85a9d (127.7.37.193:35495): tablet 3163c76cf4d6401aa70491e3fda16f68 (table test-table [id=fb5a01b4e45147eaa12b28cfac644567]) successfully deleted
I20260926 14:56:26.580708 15092 tablet_service.cc:1467] Processing CreateTablet for tablet 772fa7f1e4bf4f21ac513f7e5002df17 (DEFAULT_TABLE table=test-table [id=bb0c0fd16f9f4a8e8afbad7b9a267fbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:26.580801 15092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 772fa7f1e4bf4f21ac513f7e5002df17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:26.581017 15345 ts_tablet_manager.cc:1918] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:56:26.581055 15345 log.cc:1196] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/3163c76cf4d6401aa70491e3fda16f68
I20260926 14:56:26.581244 15345 ts_tablet_manager.cc:1939] T 3163c76cf4d6401aa70491e3fda16f68 P 0e1f78bf01a6418da2dded31464d28ac: Deleting consensus metadata
I20260926 14:56:26.581416 14800 catalog_manager.cc:4841] TS 53ad97f8c2524129bbe02a47e291ee98 (127.7.37.195:41361): tablet 3163c76cf4d6401aa70491e3fda16f68 (table test-table [id=fb5a01b4e45147eaa12b28cfac644567]) successfully deleted
I20260926 14:56:26.581538 14802 catalog_manager.cc:4841] TS 0e1f78bf01a6418da2dded31464d28ac (127.7.37.194:37885): tablet 3163c76cf4d6401aa70491e3fda16f68 (table test-table [id=fb5a01b4e45147eaa12b28cfac644567]) successfully deleted
I20260926 14:56:26.581732 15324 tablet_bootstrap.cc:492] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac: Bootstrap starting.
I20260926 14:56:26.581768 15232 tablet_service.cc:1467] Processing CreateTablet for tablet 772fa7f1e4bf4f21ac513f7e5002df17 (DEFAULT_TABLE table=test-table [id=bb0c0fd16f9f4a8e8afbad7b9a267fbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:26.581842 15232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 772fa7f1e4bf4f21ac513f7e5002df17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:26.582010 15324 tablet_bootstrap.cc:654] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:26.582470 15324 tablet_bootstrap.cc:492] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac: No bootstrap required, opened a new log
I20260926 14:56:26.582506 15324 ts_tablet_manager.cc:1397] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260926 14:56:26.582590 15324 raft_consensus.cc:348] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.582641 15324 raft_consensus.cc:374] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:26.582659 15324 raft_consensus.cc:729] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e1f78bf01a6418da2dded31464d28ac, State: Initialized, Role: FOLLOWER
I20260926 14:56:26.582703 15324 consensus_queue.cc:260] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.582787 15324 ts_tablet_manager.cc:1428] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:26.582870 15323 tablet_bootstrap.cc:492] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98: Bootstrap starting.
I20260926 14:56:26.583019 14952 tablet_service.cc:1467] Processing CreateTablet for tablet 772fa7f1e4bf4f21ac513f7e5002df17 (DEFAULT_TABLE table=test-table [id=bb0c0fd16f9f4a8e8afbad7b9a267fbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:26.583106 14952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 772fa7f1e4bf4f21ac513f7e5002df17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:26.583199 15323 tablet_bootstrap.cc:654] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:26.583658 15323 tablet_bootstrap.cc:492] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98: No bootstrap required, opened a new log
I20260926 14:56:26.583693 15323 ts_tablet_manager.cc:1397] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 14:56:26.583827 15323 raft_consensus.cc:348] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.583880 15323 raft_consensus.cc:374] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:26.583892 15323 raft_consensus.cc:729] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ad97f8c2524129bbe02a47e291ee98, State: Initialized, Role: FOLLOWER
I20260926 14:56:26.583920 15323 consensus_queue.cc:260] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.583997 15323 ts_tablet_manager.cc:1428] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:26.584122 15327 tablet_bootstrap.cc:492] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d: Bootstrap starting.
I20260926 14:56:26.584365 15327 tablet_bootstrap.cc:654] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:26.584875 15327 tablet_bootstrap.cc:492] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d: No bootstrap required, opened a new log
I20260926 14:56:26.584916 15327 ts_tablet_manager.cc:1397] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260926 14:56:26.584985 15327 raft_consensus.cc:348] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.585028 15327 raft_consensus.cc:374] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:26.585039 15327 raft_consensus.cc:729] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df829259b2534be29a2257d356c85a9d, State: Initialized, Role: FOLLOWER
I20260926 14:56:26.585067 15327 consensus_queue.cc:260] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.585141 15327 ts_tablet_manager.cc:1428] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:26.668119 15331 raft_consensus.cc:482] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:56:26.668180 15331 raft_consensus.cc:504] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.668280 15331 leader_election.cc:288] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e1f78bf01a6418da2dded31464d28ac (127.7.37.194:37885), 53ad97f8c2524129bbe02a47e291ee98 (127.7.37.195:41361)
I20260926 14:56:26.668426 15112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772fa7f1e4bf4f21ac513f7e5002df17" candidate_uuid: "df829259b2534be29a2257d356c85a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1f78bf01a6418da2dded31464d28ac" is_pre_election: true
I20260926 14:56:26.668438 15252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772fa7f1e4bf4f21ac513f7e5002df17" candidate_uuid: "df829259b2534be29a2257d356c85a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ad97f8c2524129bbe02a47e291ee98" is_pre_election: true
I20260926 14:56:26.668494 15112 raft_consensus.cc:2454] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df829259b2534be29a2257d356c85a9d in term 0.
I20260926 14:56:26.668512 15252 raft_consensus.cc:2454] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df829259b2534be29a2257d356c85a9d in term 0.
I20260926 14:56:26.668627 14906 leader_election.cc:302] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e1f78bf01a6418da2dded31464d28ac, df829259b2534be29a2257d356c85a9d; no voters:
I20260926 14:56:26.668730 15331 raft_consensus.cc:2781] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:56:26.668766 15331 raft_consensus.cc:482] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:56:26.668778 15331 raft_consensus.cc:3037] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:26.669269 15331 raft_consensus.cc:504] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.669366 15331 leader_election.cc:288] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [CANDIDATE]: Term 1 election: Requested vote from peers 0e1f78bf01a6418da2dded31464d28ac (127.7.37.194:37885), 53ad97f8c2524129bbe02a47e291ee98 (127.7.37.195:41361)
I20260926 14:56:26.669503 15252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772fa7f1e4bf4f21ac513f7e5002df17" candidate_uuid: "df829259b2534be29a2257d356c85a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ad97f8c2524129bbe02a47e291ee98"
I20260926 14:56:26.669521 15112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772fa7f1e4bf4f21ac513f7e5002df17" candidate_uuid: "df829259b2534be29a2257d356c85a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1f78bf01a6418da2dded31464d28ac"
I20260926 14:56:26.669574 15112 raft_consensus.cc:3037] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:26.669569 15252 raft_consensus.cc:3037] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:26.670114 15252 raft_consensus.cc:2454] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df829259b2534be29a2257d356c85a9d in term 1.
I20260926 14:56:26.670137 15112 raft_consensus.cc:2454] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df829259b2534be29a2257d356c85a9d in term 1.
I20260926 14:56:26.670260 14904 leader_election.cc:302] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ad97f8c2524129bbe02a47e291ee98, df829259b2534be29a2257d356c85a9d; no voters:
I20260926 14:56:26.670382 15331 raft_consensus.cc:2781] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:26.670431 15331 raft_consensus.cc:686] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [term 1 LEADER]: Becoming Leader. State: Replica: df829259b2534be29a2257d356c85a9d, State: Running, Role: LEADER
I20260926 14:56:26.670473 15331 consensus_queue.cc:237] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } }
I20260926 14:56:26.670953 14814 catalog_manager.cc:5495] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d reported cstate change: term changed from 0 to 1, leader changed from <none> to df829259b2534be29a2257d356c85a9d (127.7.37.193). New cstate: current_term: 1 leader_uuid: "df829259b2534be29a2257d356c85a9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df829259b2534be29a2257d356c85a9d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 35495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 } health_report { overall_health: UNKNOWN } } }
I20260926 14:56:26.698629 15112 raft_consensus.cc:1261] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac [term 1 FOLLOWER]: Refusing update from remote peer df829259b2534be29a2257d356c85a9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:26.698629 15252 raft_consensus.cc:1261] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98 [term 1 FOLLOWER]: Refusing update from remote peer df829259b2534be29a2257d356c85a9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:26.698797 15331 consensus_queue.cc:1035] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ad97f8c2524129bbe02a47e291ee98" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:26.698891 15353 consensus_queue.cc:1035] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e1f78bf01a6418da2dded31464d28ac" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 37885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:26.707425 14814 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:26.707491 14814 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:26.707966 14814 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ba9985f1481347e8bf5ee0656f4a6f6a: Sending DeleteTablet for 3 replicas of tablet 772fa7f1e4bf4f21ac513f7e5002df17
I20260926 14:56:26.708153 15232 tablet_service.cc:1514] Processing DeleteTablet for tablet 772fa7f1e4bf4f21ac513f7e5002df17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:26 UTC) from {username='oryx', principal='oryx/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:54984
I20260926 14:56:26.708184 14952 tablet_service.cc:1514] Processing DeleteTablet for tablet 772fa7f1e4bf4f21ac513f7e5002df17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:26 UTC) from {username='oryx', principal='oryx/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:52328
I20260926 14:56:26.708238 15347 tablet_replica.cc:330] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98: stopping tablet replica
I20260926 14:56:26.708236 15092 tablet_service.cc:1514] Processing DeleteTablet for tablet 772fa7f1e4bf4f21ac513f7e5002df17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:26 UTC) from {username='oryx', principal='oryx/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:43608
I20260926 14:56:26.708258 15346 tablet_replica.cc:330] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d: stopping tablet replica
I20260926 14:56:26.708297 15346 raft_consensus.cc:2229] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:56:26.708310 15347 raft_consensus.cc:2229] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:26.708336 15346 raft_consensus.cc:2258] T 772fa7f1e4bf4f21ac513f7e5002df17 P df829259b2534be29a2257d356c85a9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:26.708338 15347 raft_consensus.cc:2258] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:26.708407 15345 tablet_replica.cc:330] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac: stopping tablet replica
I20260926 14:56:26.708458 15345 raft_consensus.cc:2229] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:26.708492 15345 raft_consensus.cc:2258] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:26.708562 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 14887
I20260926 14:56:26.709249 15345 ts_tablet_manager.cc:1905] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:26.710197 15347 ts_tablet_manager.cc:1905] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:26.710359 15345 ts_tablet_manager.cc:1918] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 14:56:26.710415 15345 log.cc:1196] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/772fa7f1e4bf4f21ac513f7e5002df17
I20260926 14:56:26.710595 15345 ts_tablet_manager.cc:1939] T 772fa7f1e4bf4f21ac513f7e5002df17 P 0e1f78bf01a6418da2dded31464d28ac: Deleting consensus metadata
I20260926 14:56:26.710876 14802 catalog_manager.cc:4841] TS 0e1f78bf01a6418da2dded31464d28ac (127.7.37.194:37885): tablet 772fa7f1e4bf4f21ac513f7e5002df17 (table test-table [id=bb0c0fd16f9f4a8e8afbad7b9a267fbf]) successfully deleted
I20260926 14:56:26.711259 15347 ts_tablet_manager.cc:1918] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 14:56:26.711313 15347 log.cc:1196] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/772fa7f1e4bf4f21ac513f7e5002df17
I20260926 14:56:26.711467 15347 ts_tablet_manager.cc:1939] T 772fa7f1e4bf4f21ac513f7e5002df17 P 53ad97f8c2524129bbe02a47e291ee98: Deleting consensus metadata
I20260926 14:56:26.711784 14800 catalog_manager.cc:4841] TS 53ad97f8c2524129bbe02a47e291ee98 (127.7.37.195:41361): tablet 772fa7f1e4bf4f21ac513f7e5002df17 (table test-table [id=bb0c0fd16f9f4a8e8afbad7b9a267fbf]) successfully deleted
W20260926 14:56:26.714811 14799 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.37.193:35495 (error 108)
I20260926 14:56:26.715227 15404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:26.715096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:35495 (local address 127.0.0.1:52352)
0926 14:56:26.715167 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:26.715199 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.193:35495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58,"thread_start_us":36,"threads_started":1}
I20260926 14:56:26.715292 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 15027
W20260926 14:56:26.715341 14799 catalog_manager.cc:4584] TS df829259b2534be29a2257d356c85a9d (127.7.37.193:35495): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 772fa7f1e4bf4f21ac513f7e5002df17: Network error: Client connection negotiation failed: client connection to 127.7.37.193:35495: connect: Connection refused (error 111)
I20260926 14:56:26.719611 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 15167
I20260926 14:56:26.723341 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 14783
2026-09-26T14:56:26Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2392 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:56:26 dist-test-slave-f9xb krb5kdc[15407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:56:26 dist-test-slave-f9xb krb5kdc[15407](info): set up 2 sockets
Sep 26 14:56:26 dist-test-slave-f9xb krb5kdc[15407](info): commencing operation
krb5kdc: starting...
W20260926 14:56:27.659801 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.917s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:56:27 dist-test-slave-f9xb krb5kdc[15407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:56:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:56:27.766037 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:45725
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:45725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:27.840425 15420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:27.840592 15420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:27.840612 15420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:27.841709 15420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:27.841745 15420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:27.841758 15420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:27.841769 15420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:27.841786 15420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:56:27.843192 15420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:45725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:45725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:27.843355 15420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:27.843503 15420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:27.847622 15425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:27.847689 15426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:27.847678 15420 server_base.cc:1034] running on GCE node
W20260926 14:56:27.847622 15428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:27.848066 15420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:27.848318 15420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:27.849444 15420 hybrid_clock.cc:648] HybridClock initialized: now 1790434587849431 us; error 30 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-26T14:56:27Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1130 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:56:27 dist-test-slave-f9xb krb5kdc[15433](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:56:27 dist-test-slave-f9xb krb5kdc[15433](info): set up 2 sockets
Sep 26 14:56:27 dist-test-slave-f9xb krb5kdc[15433](info): commencing operation
krb5kdc: starting...
W20260926 14:56:28.795837 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:56:28 dist-test-slave-f9xb krb5kdc[15433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:56:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:56:28.898262 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:43643
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:43643
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:28.972347 15446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:28.972514 15446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:28.972534 15446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:28.973564 15446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:28.973595 15446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:28.973608 15446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:28.973619 15446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:28.973629 15446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:56:28.974947 15446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:43643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:43643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:28.975106 15446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:28.975253 15446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:28.979470 15454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:28.979501 15452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:28.979496 15451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:28.979529 15446 server_base.cc:1034] running on GCE node
I20260926 14:56:28.979862 15446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:28.980119 15446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:28.981245 15446 hybrid_clock.cc:648] HybridClock initialized: now 1790434588981230 us; error 28 us; skew 500 ppm
Sep 26 14:56:28 dist-test-slave-f9xb krb5kdc[15433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434588, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:28.983354 15446 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:56:28.983701 15446 webserver.cc:460] Webserver started at http://127.7.37.254:46069/ using document root <none> and password file <none>
I20260926 14:56:28.983917 15446 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:28.983961 15446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:28.984093 15446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:28.985193 15446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "e68a0a9717a34890b6ef7951921b6ecf"
format_stamp: "Formatted at 2026-09-26 14:56:28 on dist-test-slave-f9xb"
server_key: "4fb7abe90fe706032e535dfb836ba99b"
server_key_iv: "88f74c6c1b6164337d7cf683f2c3ac87"
server_key_version: "encryptionkey@0"
I20260926 14:56:28.985491 15446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "e68a0a9717a34890b6ef7951921b6ecf"
format_stamp: "Formatted at 2026-09-26 14:56:28 on dist-test-slave-f9xb"
server_key: "4fb7abe90fe706032e535dfb836ba99b"
server_key_iv: "88f74c6c1b6164337d7cf683f2c3ac87"
server_key_version: "encryptionkey@0"
I20260926 14:56:28.986573 15446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 14:56:28.987134 15461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:28.987275 15446 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:56:28.987341 15446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "e68a0a9717a34890b6ef7951921b6ecf"
format_stamp: "Formatted at 2026-09-26 14:56:28 on dist-test-slave-f9xb"
server_key: "4fb7abe90fe706032e535dfb836ba99b"
server_key_iv: "88f74c6c1b6164337d7cf683f2c3ac87"
server_key_version: "encryptionkey@0"
I20260926 14:56:28.987401 15446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:29.002007 15446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:29.003842 15446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:29.003960 15446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:29.007266 15446 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:43643
I20260926 14:56:29.007297 15513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:43643 every 8 connection(s)
I20260926 14:56:29.007560 15446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:56:29.008311 15514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:29.010253 15514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf: Bootstrap starting.
I20260926 14:56:29.010748 15514 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:29.010995 15514 log.cc:824] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:29.011588 15514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf: No bootstrap required, opened a new log
I20260926 14:56:29.011575 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 15446
I20260926 14:56:29.011691 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:56:29.011866 7319 external_mini_cluster.cc:1428] Setting key 659d81c325cd2c29047977d1a94183b1
I20260926 14:56:29.013566 15514 raft_consensus.cc:348] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68a0a9717a34890b6ef7951921b6ecf" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43643 } }
I20260926 14:56:29.013720 15514 raft_consensus.cc:374] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:29.013773 15514 raft_consensus.cc:729] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e68a0a9717a34890b6ef7951921b6ecf, State: Initialized, Role: FOLLOWER
I20260926 14:56:29.013896 15514 consensus_queue.cc:260] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68a0a9717a34890b6ef7951921b6ecf" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43643 } }
I20260926 14:56:29.013967 15514 raft_consensus.cc:388] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:56:29.014001 15514 raft_consensus.cc:482] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:56:29.014043 15514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:29.014626 15514 raft_consensus.cc:504] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68a0a9717a34890b6ef7951921b6ecf" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43643 } }
I20260926 14:56:29.014730 15514 leader_election.cc:302] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e68a0a9717a34890b6ef7951921b6ecf; no voters:
I20260926 14:56:29.014868 15514 leader_election.cc:288] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:56:29.014901 15519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:29.015056 15514 sys_catalog.cc:564] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:29.015044 15519 raft_consensus.cc:686] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [term 1 LEADER]: Becoming Leader. State: Replica: e68a0a9717a34890b6ef7951921b6ecf, State: Running, Role: LEADER
I20260926 14:56:29.016140 15519 consensus_queue.cc:237] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68a0a9717a34890b6ef7951921b6ecf" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43643 } }
I20260926 14:56:29.016613 15521 sys_catalog.cc:455] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e68a0a9717a34890b6ef7951921b6ecf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68a0a9717a34890b6ef7951921b6ecf" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43643 } } }
W20260926 14:56:29.017308 15533 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260926 14:56:29.016654 15520 sys_catalog.cc:455] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [sys.catalog]: SysCatalogTable state changed. Reason: New leader e68a0a9717a34890b6ef7951921b6ecf. Latest consensus state: current_term: 1 leader_uuid: "e68a0a9717a34890b6ef7951921b6ecf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68a0a9717a34890b6ef7951921b6ecf" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43643 } } }
W20260926 14:56:29.017352 15533 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 14:56:29.017364 15520 sys_catalog.cc:458] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:29.017360 15521 sys_catalog.cc:458] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:29.018311 15536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:56:29.018492 15536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:29.019841 15536 catalog_manager.cc:1347] Generated new cluster ID: 80abfef18c4a43498425ca4b8e360f96
I20260926 14:56:29.019891 15536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:56:29.022234 15518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:29.012279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48814 (local address 127.7.37.254:43643)
0926 14:56:29.012420 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:29.012428 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:29.012462 (+ 34us) server_negotiation.cc:407] Connection header received
0926 14:56:29.012860 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:29.012868 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:29.012924 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:29.013061 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:29.013589 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.015148 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.016236 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.016748 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.016841 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:29.019200 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:29.019211 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:29.019212 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:29.019222 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:29.020809 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:29.021110 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:29.021113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:29.021113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:29.021193 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:29.021362 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:29.021363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:29.021364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:29.021571 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:29.021711 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:29.021814 (+ 103us) server_negotiation.cc:299] Negotiation successful
0926 14:56:29.021934 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":67,"threads_started":1}
I20260926 14:56:29.069137 15536 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:56:29.069816 15536 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:29.074640 15536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf: Generated new TSK 0
I20260926 14:56:29.074787 15536 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
I20260926 14:56:29.104756 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:43643
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:29.181133 15540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:29.181321 15540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:29.181350 15540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:29.182461 15540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:29.182500 15540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:29.182533 15540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:56:29.183939 15540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:43643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:29.184165 15540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:29.184360 15540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:29.188655 15546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:29.188760 15540 server_base.cc:1034] running on GCE node
W20260926 14:56:29.188722 15545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:29.188681 15548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:29.189036 15540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:29.189256 15540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:29.190389 15540 hybrid_clock.cc:648] HybridClock initialized: now 1790434589190376 us; error 27 us; skew 500 ppm
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434589, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:29.192744 15540 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:56:29.193100 15540 webserver.cc:460] Webserver started at http://127.7.37.193:40025/ using document root <none> and password file <none>
I20260926 14:56:29.193282 15540 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:29.193325 15540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:29.193439 15540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:29.194336 15540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "d52eca44e72647f1a7b2c458b5c4554e"
format_stamp: "Formatted at 2026-09-26 14:56:29 on dist-test-slave-f9xb"
server_key: "4730c78d592f41c0ca8f14278a2a4a47"
server_key_iv: "76a830001d4a96b8d99aeec892386a38"
server_key_version: "encryptionkey@0"
I20260926 14:56:29.194630 15540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "d52eca44e72647f1a7b2c458b5c4554e"
format_stamp: "Formatted at 2026-09-26 14:56:29 on dist-test-slave-f9xb"
server_key: "4730c78d592f41c0ca8f14278a2a4a47"
server_key_iv: "76a830001d4a96b8d99aeec892386a38"
server_key_version: "encryptionkey@0"
I20260926 14:56:29.195725 15540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 14:56:29.196338 15555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:29.196475 15540 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:29.196547 15540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "d52eca44e72647f1a7b2c458b5c4554e"
format_stamp: "Formatted at 2026-09-26 14:56:29 on dist-test-slave-f9xb"
server_key: "4730c78d592f41c0ca8f14278a2a4a47"
server_key_iv: "76a830001d4a96b8d99aeec892386a38"
server_key_version: "encryptionkey@0"
I20260926 14:56:29.196602 15540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:29.203505 15540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:29.205034 15540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:29.205147 15540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:29.205307 15540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:29.205541 15540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:29.205570 15540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:29.205595 15540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:29.205618 15540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:29.210806 15540 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:45615
I20260926 14:56:29.210856 15668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:45615 every 8 connection(s)
I20260926 14:56:29.211104 15540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434589, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:29.218722 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 15540
I20260926 14:56:29.218824 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:56:29.218982 7319 external_mini_cluster.cc:1428] Setting key 6d1aeda773056beae0a53e0da000606d
I20260926 14:56:29.219432 15518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:29.211671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:34037 (local address 127.7.37.254:43643)
0926 14:56:29.211716 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:29.211718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:29.211927 (+ 209us) server_negotiation.cc:407] Connection header received
0926 14:56:29.212368 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:29.212369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:29.212382 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:29.212427 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:29.213234 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.213727 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.214622 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.214848 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.214905 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:29.216844 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:29.216845 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:29.216845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:29.216852 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:29.218164 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:29.218476 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:29.218477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:29.218477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:29.218523 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:29.218762 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:29.218762 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:29.218762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:29.218886 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:29.218941 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:29.219257 (+ 316us) server_negotiation.cc:299] Negotiation successful
0926 14:56:29.219338 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 14:56:29.219466 15671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:29.211657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.193:34037)
0926 14:56:29.211826 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:29.211857 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:29.212280 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:29.212495 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:29.212499 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:29.212990 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:29.213150 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:29.213153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.213800 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.213801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:29.214549 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:29.214551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.214919 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.214920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:29.215028 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:29.215419 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:29.215426 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:29.216761 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:29.218245 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:29.218248 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:29.218249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:29.218426 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:29.218573 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:29.218573 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:29.218574 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:29.218701 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:29.218950 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:29.218955 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:29.219098 (+ 143us) client_negotiation.cc:769] Sending connection context
0926 14:56:29.219192 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 14:56:29.219334 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260926 14:56:29.219926 15669 heartbeater.cc:347] Connected to a master server at 127.7.37.254:43643
I20260926 14:56:29.220006 15669 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:29.220160 15669 heartbeater.cc:511] Master 127.7.37.254:43643 requested a full tablet report, sending...
I20260926 14:56:29.220628 15478 ts_manager.cc:194] Registered new tserver with Master: d52eca44e72647f1a7b2c458b5c4554e (127.7.37.193:45615)
I20260926 14:56:29.221314 15478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:34037
WARNING: no policy specified for kudu/127.7.37.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
I20260926 14:56:29.240280 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:43643
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:29.317811 15674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:29.317977 15674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:29.317998 15674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:29.319061 15674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:29.319093 15674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:29.319116 15674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:56:29.320448 15674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:43643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:29.320601 15674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:29.320761 15674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:29.324958 15682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:29.325083 15674 server_base.cc:1034] running on GCE node
W20260926 14:56:29.325002 15679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:29.324963 15680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:29.325397 15674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:29.325598 15674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:29.326735 15674 hybrid_clock.cc:648] HybridClock initialized: now 1790434589326718 us; error 27 us; skew 500 ppm
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434589, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:29.329032 15674 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:56:29.329423 15674 webserver.cc:460] Webserver started at http://127.7.37.194:35657/ using document root <none> and password file <none>
I20260926 14:56:29.329612 15674 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:29.329658 15674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:29.329780 15674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:29.330668 15674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "dd66611745484d11998116cc55e9e2be"
format_stamp: "Formatted at 2026-09-26 14:56:29 on dist-test-slave-f9xb"
server_key: "0301e113e254cd1658f34b3e5afa0cf9"
server_key_iv: "22623de70bd41da696c0d90a4572be8f"
server_key_version: "encryptionkey@0"
I20260926 14:56:29.330974 15674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "dd66611745484d11998116cc55e9e2be"
format_stamp: "Formatted at 2026-09-26 14:56:29 on dist-test-slave-f9xb"
server_key: "0301e113e254cd1658f34b3e5afa0cf9"
server_key_iv: "22623de70bd41da696c0d90a4572be8f"
server_key_version: "encryptionkey@0"
I20260926 14:56:29.332124 15674 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 14:56:29.332799 15689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:29.332948 15674 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:56:29.333017 15674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "dd66611745484d11998116cc55e9e2be"
format_stamp: "Formatted at 2026-09-26 14:56:29 on dist-test-slave-f9xb"
server_key: "0301e113e254cd1658f34b3e5afa0cf9"
server_key_iv: "22623de70bd41da696c0d90a4572be8f"
server_key_version: "encryptionkey@0"
I20260926 14:56:29.333072 15674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:29.346720 15674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:29.348610 15674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:29.348730 15674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:29.348901 15674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:29.349140 15674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:29.349171 15674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:29.349195 15674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:29.349212 15674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:29.354223 15674 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:43013
I20260926 14:56:29.354279 15802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:43013 every 8 connection(s)
I20260926 14:56:29.354508 15674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434589, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:29.361828 15518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:29.355095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:41055 (local address 127.7.37.254:43643)
0926 14:56:29.355154 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:29.355156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:29.355304 (+ 148us) server_negotiation.cc:407] Connection header received
0926 14:56:29.355679 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:29.355681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:29.355689 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:29.355736 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:29.356528 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.356994 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.357882 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.358054 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.358098 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:29.359851 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:29.359853 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:29.359853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:29.359858 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:29.360868 (+ 1010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:29.361167 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:29.361168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:29.361168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:29.361207 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:29.361403 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:29.361404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:29.361404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:29.361498 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:29.361543 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:29.361721 (+ 178us) server_negotiation.cc:299] Negotiation successful
0926 14:56:29.361771 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 14:56:29.361970 15805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:29.355060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.194:41055)
0926 14:56:29.355187 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:29.355222 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:29.355614 (+ 392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:29.355815 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:29.355818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:29.356301 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:29.356465 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:29.356469 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.357052 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.357052 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:29.357821 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:29.357823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.358105 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.358105 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:29.358188 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:29.358581 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:29.358590 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:29.359775 (+ 1185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:29.360924 (+ 1149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:29.360928 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:29.360930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:29.361113 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:29.361253 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:29.361254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:29.361254 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:29.361354 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:29.361541 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:29.361543 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:29.361635 (+ 92us) client_negotiation.cc:769] Sending connection context
0926 14:56:29.361702 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 14:56:29.361800 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":68,"threads_started":1}
I20260926 14:56:29.362383 15803 heartbeater.cc:347] Connected to a master server at 127.7.37.254:43643
I20260926 14:56:29.362455 15803 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:29.362569 15803 heartbeater.cc:511] Master 127.7.37.254:43643 requested a full tablet report, sending...
I20260926 14:56:29.362861 15478 ts_manager.cc:194] Registered new tserver with Master: dd66611745484d11998116cc55e9e2be (127.7.37.194:43013)
I20260926 14:56:29.363174 15478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:41055
I20260926 14:56:29.363873 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 15674
I20260926 14:56:29.363958 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:56:29.364104 7319 external_mini_cluster.cc:1428] Setting key 292bcb39c87ee73c72d9611470d026d3
WARNING: no policy specified for kudu/127.7.37.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
I20260926 14:56:29.383922 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:43643
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:29.457882 15808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:29.458046 15808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:29.458076 15808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:29.459174 15808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:29.459213 15808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:29.459245 15808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:56:29.460624 15808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:43643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:29.460841 15808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:29.461026 15808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:29.465363 15814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:29.465387 15816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:29.465461 15808 server_base.cc:1034] running on GCE node
W20260926 14:56:29.465523 15813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:29.465723 15808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:29.465927 15808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:29.467056 15808 hybrid_clock.cc:648] HybridClock initialized: now 1790434589467049 us; error 30 us; skew 500 ppm
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434589, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:29.469198 15808 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:56:29.469554 15808 webserver.cc:460] Webserver started at http://127.7.37.195:43247/ using document root <none> and password file <none>
I20260926 14:56:29.469738 15808 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:29.469789 15808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:29.469898 15808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:29.470757 15808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "5831683ff1a6492aa14f925ee99ed78a"
format_stamp: "Formatted at 2026-09-26 14:56:29 on dist-test-slave-f9xb"
server_key: "46c41e2242b11633395f78a471dba464"
server_key_iv: "6f4dac29e6b220a6399e4edae9d9d795"
server_key_version: "encryptionkey@0"
I20260926 14:56:29.471091 15808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "5831683ff1a6492aa14f925ee99ed78a"
format_stamp: "Formatted at 2026-09-26 14:56:29 on dist-test-slave-f9xb"
server_key: "46c41e2242b11633395f78a471dba464"
server_key_iv: "6f4dac29e6b220a6399e4edae9d9d795"
server_key_version: "encryptionkey@0"
I20260926 14:56:29.472257 15808 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 14:56:29.472859 15823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:29.472997 15808 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:29.473068 15808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "5831683ff1a6492aa14f925ee99ed78a"
format_stamp: "Formatted at 2026-09-26 14:56:29 on dist-test-slave-f9xb"
server_key: "46c41e2242b11633395f78a471dba464"
server_key_iv: "6f4dac29e6b220a6399e4edae9d9d795"
server_key_version: "encryptionkey@0"
I20260926 14:56:29.473129 15808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:29.480324 15808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:29.481739 15808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:29.481860 15808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:29.482021 15808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:29.482247 15808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:29.482276 15808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:29.482301 15808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:29.482326 15808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:29.487329 15808 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:39969
I20260926 14:56:29.487382 15936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:39969 every 8 connection(s)
I20260926 14:56:29.487613 15808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434589, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:29.495172 15518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:29.488249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:45137 (local address 127.7.37.254:43643)
0926 14:56:29.488287 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:29.488290 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:29.488452 (+ 162us) server_negotiation.cc:407] Connection header received
0926 14:56:29.488846 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:29.488846 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:29.488855 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:29.488898 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:29.489714 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.490187 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.491128 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.491314 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.491357 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:29.493069 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:29.493070 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:29.493070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:29.493076 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:29.494069 (+ 993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:29.494400 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:29.494401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:29.494402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:29.494448 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:29.494638 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:29.494639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:29.494639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:29.494735 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:29.494785 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:29.495017 (+ 232us) server_negotiation.cc:299] Negotiation successful
0926 14:56:29.495087 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 14:56:29.495307 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:29.488219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.195:45137)
0926 14:56:29.488353 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:29.488389 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:29.488778 (+ 389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:29.488950 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:29.488952 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:29.489488 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:29.489653 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:29.489657 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.490245 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.490245 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:29.491063 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:29.491065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.491367 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.491368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:29.491462 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:29.491824 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:29.491832 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:29.492982 (+ 1150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:29.494136 (+ 1154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:29.494141 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:29.494143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:29.494339 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:29.494500 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:29.494501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:29.494501 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:29.494585 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:29.494785 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:29.494790 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:29.494908 (+ 118us) client_negotiation.cc:769] Sending connection context
0926 14:56:29.494987 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 14:56:29.495119 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":63,"threads_started":1}
I20260926 14:56:29.495656 15937 heartbeater.cc:347] Connected to a master server at 127.7.37.254:43643
I20260926 14:56:29.495729 15937 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:29.495884 15937 heartbeater.cc:511] Master 127.7.37.254:43643 requested a full tablet report, sending...
I20260926 14:56:29.496184 15478 ts_manager.cc:194] Registered new tserver with Master: 5831683ff1a6492aa14f925ee99ed78a (127.7.37.195:39969)
I20260926 14:56:29.496491 15478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:45137
I20260926 14:56:29.497222 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 15808
I20260926 14:56:29.497301 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:56:29.497422 7319 external_mini_cluster.cc:1428] Setting key 6cee3408689b3c191375528e5bf18e4e
I20260926 14:56:29.498142 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.193.keytab.
Entry for principal oryx/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.193.keytab.
WARNING: no policy specified for oryx/127.7.37.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.37.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.194.keytab.
Entry for principal oryx/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.194.keytab.
WARNING: no policy specified for oryx/127.7.37.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.37.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.195.keytab.
Entry for principal oryx/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.195.keytab.
I20260926 14:56:29.555554 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 15540
I20260926 14:56:29.559315 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 15674
I20260926 14:56:29.562754 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 15808
I20260926 14:56:29.566105 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 15446
I20260926 14:56:29.569972 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:43643
--webserver_interface=127.7.37.254
--webserver_port=46069
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:43643
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:29.643955 15946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:29.644120 15946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:29.644150 15946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:29.645217 15946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:29.645255 15946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:29.645283 15946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:29.645299 15946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:29.645321 15946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:56:29.646670 15946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:43643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:43643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=46069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:29.646876 15946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:29.647059 15946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:29.651249 15952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:29.651293 15951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:29.651301 15954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:29.651429 15946 server_base.cc:1034] running on GCE node
I20260926 14:56:29.651619 15946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:29.651895 15946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:29.653023 15946 hybrid_clock.cc:648] HybridClock initialized: now 1790434589653008 us; error 26 us; skew 500 ppm
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434589, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:29.655184 15946 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:56:29.655545 15946 webserver.cc:460] Webserver started at http://127.7.37.254:46069/ using document root <none> and password file <none>
I20260926 14:56:29.655716 15946 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:29.655791 15946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:29.656932 15946 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 14:56:29.657435 15961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:29.657574 15946 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:56:29.657645 15946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "e68a0a9717a34890b6ef7951921b6ecf"
format_stamp: "Formatted at 2026-09-26 14:56:28 on dist-test-slave-f9xb"
server_key: "4fb7abe90fe706032e535dfb836ba99b"
server_key_iv: "88f74c6c1b6164337d7cf683f2c3ac87"
server_key_version: "encryptionkey@0"
I20260926 14:56:29.657908 15946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:29.698097 15946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:29.700080 15946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:29.700202 15946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:29.703558 15946 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:43643
I20260926 14:56:29.703591 16013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:43643 every 8 connection(s)
I20260926 14:56:29.703912 15946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:56:29.705168 16014 sys_catalog.cc:263] Verifying existing consensus state
I20260926 14:56:29.705765 16014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf: Bootstrap starting.
I20260926 14:56:29.713069 16014 log.cc:824] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:29.713497 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 15946
I20260926 14:56:29.713770 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:45615
--local_ip_for_outbound_sockets=127.7.37.193
--tserver_master_addrs=127.7.37.254:43643
--webserver_port=40025
--webserver_interface=127.7.37.193
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.37.193
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 14:56:29.714741 16014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 14:56:29.715000 16014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf: Bootstrap complete.
I20260926 14:56:29.716955 16014 raft_consensus.cc:348] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68a0a9717a34890b6ef7951921b6ecf" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43643 } }
I20260926 14:56:29.717157 16014 raft_consensus.cc:729] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e68a0a9717a34890b6ef7951921b6ecf, State: Initialized, Role: FOLLOWER
I20260926 14:56:29.717281 16014 consensus_queue.cc:260] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68a0a9717a34890b6ef7951921b6ecf" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43643 } }
I20260926 14:56:29.717350 16014 raft_consensus.cc:388] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:56:29.717389 16014 raft_consensus.cc:482] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:56:29.717430 16014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [term 1 FOLLOWER]: Advancing to term 2
I20260926 14:56:29.718284 16014 raft_consensus.cc:504] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68a0a9717a34890b6ef7951921b6ecf" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43643 } }
I20260926 14:56:29.718371 16014 leader_election.cc:302] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e68a0a9717a34890b6ef7951921b6ecf; no voters:
I20260926 14:56:29.718511 16014 leader_election.cc:288] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 14:56:29.718559 16018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [term 2 FOLLOWER]: Leader election won for term 2
I20260926 14:56:29.718715 16018 raft_consensus.cc:686] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [term 2 LEADER]: Becoming Leader. State: Replica: e68a0a9717a34890b6ef7951921b6ecf, State: Running, Role: LEADER
I20260926 14:56:29.718740 16014 sys_catalog.cc:564] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:29.718873 16018 consensus_queue.cc:237] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68a0a9717a34890b6ef7951921b6ecf" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43643 } }
I20260926 14:56:29.719134 16019 sys_catalog.cc:455] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e68a0a9717a34890b6ef7951921b6ecf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68a0a9717a34890b6ef7951921b6ecf" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43643 } } }
I20260926 14:56:29.719206 16019 sys_catalog.cc:458] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:29.719350 16020 sys_catalog.cc:455] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [sys.catalog]: SysCatalogTable state changed. Reason: New leader e68a0a9717a34890b6ef7951921b6ecf. Latest consensus state: current_term: 2 leader_uuid: "e68a0a9717a34890b6ef7951921b6ecf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68a0a9717a34890b6ef7951921b6ecf" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43643 } } }
I20260926 14:56:29.719403 16020 sys_catalog.cc:458] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:29.719496 16023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:56:29.719949 16023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:56:29.720065 16023 catalog_manager.cc:1259] Loaded cluster ID: 80abfef18c4a43498425ca4b8e360f96
I20260926 14:56:29.720247 16023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:56:29.721458 16023 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:29.721619 16023 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e68a0a9717a34890b6ef7951921b6ecf: Loaded TSK: 0
I20260926 14:56:29.721832 16023 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 14:56:29.789146 16017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:29.789314 16017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:29.789335 16017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:29.790426 16017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:29.790462 16017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:29.790485 16017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:56:29.791846 16017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.193.keytab
--principal=oryx/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:45615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=40025
--tserver_master_addrs=127.7.37.254:43643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:29.792038 16017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:29.792222 16017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:29.798593 16041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:29.798699 16017 server_base.cc:1034] running on GCE node
W20260926 14:56:29.798595 16043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:29.798884 16040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:29.799070 16017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:29.799271 16017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:29.800395 16017 hybrid_clock.cc:648] HybridClock initialized: now 1790434589800383 us; error 29 us; skew 500 ppm
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434589, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:29.802610 16017 init.cc:377] Logged in from keytab as oryx/127.7.37.193@KRBTEST.COM (short username oryx)
I20260926 14:56:29.802997 16017 webserver.cc:460] Webserver started at http://127.7.37.193:40025/ using document root <none> and password file <none>
I20260926 14:56:29.803179 16017 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:29.803236 16017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:29.804452 16017 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260926 14:56:29.804967 16050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:29.805111 16017 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:29.805182 16017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "d52eca44e72647f1a7b2c458b5c4554e"
format_stamp: "Formatted at 2026-09-26 14:56:29 on dist-test-slave-f9xb"
server_key: "4730c78d592f41c0ca8f14278a2a4a47"
server_key_iv: "76a830001d4a96b8d99aeec892386a38"
server_key_version: "encryptionkey@0"
I20260926 14:56:29.805450 16017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:29.816903 16017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:29.818715 16017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:29.818835 16017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:29.819005 16017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:29.819267 16017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:29.819306 16017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:29.819344 16017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:29.819368 16017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:29.824733 16017 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:45615
I20260926 14:56:29.824786 16163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:45615 every 8 connection(s)
I20260926 14:56:29.825042 16017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:56:29.828130 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 16017
I20260926 14:56:29.828368 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:43013
--local_ip_for_outbound_sockets=127.7.37.194
--tserver_master_addrs=127.7.37.254:43643
--webserver_port=35657
--webserver_interface=127.7.37.194
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.37.194
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:29.832882 16166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:29.825638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.193:52229)
0926 14:56:29.825771 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:29.825806 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:29.826177 (+ 371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:29.826558 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:29.826561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:29.827030 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:29.827188 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:29.827192 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.828765 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.828766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:29.829780 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:29.829782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.830304 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.830305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:29.830422 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:29.830804 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:29.830811 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:29.832703 (+ 1892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":64,"threads_started":1}
W20260926 14:56:29.832955 16167 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:29.825707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:52229 (local address 127.7.37.254:43643)
0926 14:56:29.825876 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:29.825883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:29.825912 (+ 29us) server_negotiation.cc:407] Connection header received
0926 14:56:29.826343 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:29.826349 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:29.826400 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:29.826527 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:29.827295 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.828673 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.829857 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.830235 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.830322 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:29.832817 (+ 2495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.193:52229: BlockingRecv error: recv got EOF from 127.7.37.193:52229 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
W20260926 14:56:29.833292 16164 heartbeater.cc:644] Failed to heartbeat to 127.7.37.254:43643 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.37.254:43643: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:29.836970 16166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:29.833467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.193:34091)
0926 14:56:29.833499 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:29.833508 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:29.833539 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:29.833652 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:29.833652 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:29.833759 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:29.833797 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:29.833798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.834359 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.834360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:29.834838 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:29.834839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.835127 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.835128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:29.835186 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:29.835523 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:29.835528 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:29.836917 (+ 1389us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":21}
W20260926 14:56:29.837044 16167 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:29.833498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:34091 (local address 127.7.37.254:43643)
0926 14:56:29.833526 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:29.833528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:29.833535 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:29.833589 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:29.833590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:29.833600 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:29.833645 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:29.833862 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.834293 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.834899 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.835075 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.835125 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:29.836996 (+ 1871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.193:34091: BlockingRecv error: recv got EOF from 127.7.37.193:34091 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:29.840817 16166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:29.837330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.193:45117)
0926 14:56:29.837361 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:29.837369 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:29.837400 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:29.837505 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:29.837506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:29.837615 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:29.837635 (+ 20us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:29.837636 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.838205 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.838206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:29.838681 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:29.838682 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.839063 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.839064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:29.839114 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:29.839417 (+ 303us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:29.839421 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:29.840765 (+ 1344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":20}
W20260926 14:56:29.840968 16164 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 14:56:29.840922 16167 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:29.837373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:45117 (local address 127.7.37.254:43643)
0926 14:56:29.837401 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:29.837403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:29.837409 (+ 6us) server_negotiation.cc:407] Connection header received
0926 14:56:29.837445 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:29.837446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:29.837454 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:29.837502 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:29.837690 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.838145 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.838757 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.838999 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.839045 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:29.840872 (+ 1827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.193:45117: BlockingRecv error: recv got EOF from 127.7.37.193:45117 (error 108)
Metrics: {"server-negotiator.queue_time_us":13}
W20260926 14:56:29.905978 16168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:29.906136 16168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:29.906157 16168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:29.907249 16168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:29.907287 16168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:29.907313 16168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:56:29.908757 16168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.194.keytab
--principal=oryx/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:43013
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=35657
--tserver_master_addrs=127.7.37.254:43643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:29.908955 16168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:29.909134 16168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:29.913574 16174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:29.913663 16176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:29.913578 16173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:29.913712 16168 server_base.cc:1034] running on GCE node
I20260926 14:56:29.913902 16168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:29.914106 16168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:29.915242 16168 hybrid_clock.cc:648] HybridClock initialized: now 1790434589915226 us; error 33 us; skew 500 ppm
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434589, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:29.917340 16168 init.cc:377] Logged in from keytab as oryx/127.7.37.194@KRBTEST.COM (short username oryx)
I20260926 14:56:29.917701 16168 webserver.cc:460] Webserver started at http://127.7.37.194:35657/ using document root <none> and password file <none>
I20260926 14:56:29.917894 16168 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:29.917953 16168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:29.919101 16168 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:29.919606 16183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:29.919760 16168 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:29.919850 16168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "dd66611745484d11998116cc55e9e2be"
format_stamp: "Formatted at 2026-09-26 14:56:29 on dist-test-slave-f9xb"
server_key: "0301e113e254cd1658f34b3e5afa0cf9"
server_key_iv: "22623de70bd41da696c0d90a4572be8f"
server_key_version: "encryptionkey@0"
I20260926 14:56:29.920109 16168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:29.933578 16168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:29.935012 16168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:29.935125 16168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:29.935294 16168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:29.935521 16168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:29.935552 16168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:29.935576 16168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:29.935599 16168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:29.940909 16168 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:43013
I20260926 14:56:29.940950 16296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:43013 every 8 connection(s)
I20260926 14:56:29.941213 16168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:56:29.942987 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 16168
I20260926 14:56:29.943212 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:39969
--local_ip_for_outbound_sockets=127.7.37.195
--tserver_master_addrs=127.7.37.254:43643
--webserver_port=43247
--webserver_interface=127.7.37.195
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.37.195
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 14:56:29.947770 16167 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:29.941804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:34391 (local address 127.7.37.254:43643)
0926 14:56:29.941869 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:29.941871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:29.941963 (+ 92us) server_negotiation.cc:407] Connection header received
0926 14:56:29.942359 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:29.942360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:29.942371 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:29.942415 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:29.943214 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.943920 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.944818 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.945090 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.945160 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:29.947697 (+ 2537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.194:34391: BlockingRecv error: recv got EOF from 127.7.37.194:34391 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 14:56:29.947829 16299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:29.941749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.194:34391)
0926 14:56:29.941876 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:29.941906 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:29.942289 (+ 383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:29.942459 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:29.942462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:29.942956 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:29.943139 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:29.943143 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.943992 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.943993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:29.944744 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:29.944746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.945158 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.945159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:29.945254 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:29.945629 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:29.945637 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:29.947612 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
W20260926 14:56:29.948006 16297 heartbeater.cc:644] Failed to heartbeat to 127.7.37.254:43643 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.37.254:43643: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:29.951838 16299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:29.948168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.194:60863)
0926 14:56:29.948256 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:29.948265 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:29.948307 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:29.948411 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:29.948411 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:29.948521 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:29.948557 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:29.948558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.949104 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.949105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:29.949586 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:29.949588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.949886 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.949887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:29.949948 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:29.950266 (+ 318us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:29.950271 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:29.951779 (+ 1508us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":75}
W20260926 14:56:29.951889 16167 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:29.948221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:60863 (local address 127.7.37.254:43643)
0926 14:56:29.948261 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:29.948264 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:29.948302 (+ 38us) server_negotiation.cc:407] Connection header received
0926 14:56:29.948360 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:29.948360 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:29.948368 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:29.948410 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:29.948614 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.949045 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.949648 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.949835 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.949884 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:29.951848 (+ 1964us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.194:60863: BlockingRecv error: recv got EOF from 127.7.37.194:60863 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:29 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:29.955643 16299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:29.952172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.194:56005)
0926 14:56:29.952211 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:29.952220 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:29.952250 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:29.952356 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:29.952357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:29.952463 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:29.952484 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:29.952485 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.953025 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.953026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:29.953499 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:29.953500 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.953803 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.953804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:29.953852 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:29.954157 (+ 305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:29.954161 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:29.955593 (+ 1432us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260926 14:56:29.955758 16167 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:29.952230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:56005 (local address 127.7.37.254:43643)
0926 14:56:29.952272 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:29.952274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:29.952281 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:29.952305 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:29.952306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:29.952313 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:29.952355 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:29.952537 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.952967 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.953559 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:29.953745 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:29.953793 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:29.955661 (+ 1868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.194:56005: BlockingRecv error: recv got EOF from 127.7.37.194:56005 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260926 14:56:29.955832 16297 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 14:56:30.019933 16300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:30.020099 16300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:30.020119 16300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:30.021278 16300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:30.021327 16300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:30.021354 16300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:56:30.022904 16300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/krb5kdc/oryx_127.7.37.195.keytab
--principal=oryx/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:39969
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=43247
--tserver_master_addrs=127.7.37.254:43643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:30.023058 16300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:30.023222 16300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:30.027534 16308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:30.027612 16306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:30.027524 16305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:30.027661 16300 server_base.cc:1034] running on GCE node
I20260926 14:56:30.027858 16300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:30.028054 16300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:30.029222 16300 hybrid_clock.cc:648] HybridClock initialized: now 1790434590029200 us; error 38 us; skew 500 ppm
Sep 26 14:56:30 dist-test-slave-f9xb krb5kdc[15433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434590, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:30.031435 16300 init.cc:377] Logged in from keytab as oryx/127.7.37.195@KRBTEST.COM (short username oryx)
I20260926 14:56:30.031860 16300 webserver.cc:460] Webserver started at http://127.7.37.195:43247/ using document root <none> and password file <none>
I20260926 14:56:30.032040 16300 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:30.032097 16300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:30.033236 16300 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:30.033754 16315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:30.033902 16300 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:30.033984 16300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "5831683ff1a6492aa14f925ee99ed78a"
format_stamp: "Formatted at 2026-09-26 14:56:29 on dist-test-slave-f9xb"
server_key: "46c41e2242b11633395f78a471dba464"
server_key_iv: "6f4dac29e6b220a6399e4edae9d9d795"
server_key_version: "encryptionkey@0"
I20260926 14:56:30.034303 16300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:30.047282 16300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:30.049124 16300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:30.049247 16300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:30.049409 16300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:30.049654 16300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:30.049685 16300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:30.049702 16300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:30.049714 16300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:30.054755 16300 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:39969
I20260926 14:56:30.054833 16428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:39969 every 8 connection(s)
I20260926 14:56:30.055078 16300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:56:30.057718 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 16300
Sep 26 14:56:30 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:30 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:30.061833 16431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:30.055639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.195:46645)
0926 14:56:30.055791 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:30.055828 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:30.056248 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:30.056452 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:30.056455 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:30.056948 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:30.057114 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:30.057118 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.057887 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.057888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:30.058770 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:30.058773 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.059147 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.059147 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:30.059275 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:30.059691 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:30.059698 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:30.061705 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
W20260926 14:56:30.061837 16167 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:30.055657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:46645 (local address 127.7.37.254:43643)
0926 14:56:30.055695 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:30.055697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:30.055898 (+ 201us) server_negotiation.cc:407] Connection header received
0926 14:56:30.056333 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:30.056335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:30.056346 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:30.056395 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:30.057200 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.057785 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.058847 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.059073 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.059120 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:30.061779 (+ 2659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.195:46645: BlockingRecv error: recv got EOF from 127.7.37.195:46645 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260926 14:56:30.062016 16429 heartbeater.cc:644] Failed to heartbeat to 127.7.37.254:43643 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.37.254:43643: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
I20260926 14:56:30.064955 16433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:30.058156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48816 (local address 127.7.37.254:43643)
0926 14:56:30.058264 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:30.058266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:30.058273 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:30.058292 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:30.058293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:30.058302 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:30.058356 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:30.058884 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.059468 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.060261 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.060462 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.060513 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:30.061617 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:30.061621 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:30.061621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:30.061632 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:30.063359 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:30.064022 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:30.064024 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:30.064025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:30.064107 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:30.064297 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:30.064298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:30.064299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:30.064497 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:30.064607 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:30.064719 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 14:56:30.064895 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":51,"threads_started":1}
Sep 26 14:56:30 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:30 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:30.066223 16431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:30.062218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.195:43013)
0926 14:56:30.062247 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:30.062256 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:30.062291 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:30.062430 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:30.062431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:30.062559 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:30.062603 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:30.062605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.063410 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.063411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:30.063933 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:30.063934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.064241 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.064242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:30.064317 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:30.064634 (+ 317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:30.064639 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:30.066168 (+ 1529us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":16}
W20260926 14:56:30.066295 16167 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:30.062296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:43013 (local address 127.7.37.254:43643)
0926 14:56:30.062331 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:30.062333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:30.062340 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:30.062358 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:30.062358 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:30.062366 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:30.062417 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:30.062673 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.063346 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.064000 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.064184 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.064243 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:30.066251 (+ 2008us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.195:43013: BlockingRecv error: recv got EOF from 127.7.37.195:43013 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Sep 26 14:56:30 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:30 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:30.069690 16431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:30.066410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.195:43031)
0926 14:56:30.066507 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:30.066516 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:30.066570 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:30.066658 (+ 88us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:30.066659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:30.066756 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:30.066779 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:30.066779 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.067269 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.067269 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:30.067708 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:30.067709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.068018 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.068018 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:30.068052 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:30.068365 (+ 313us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:30.068369 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:30.069652 (+ 1283us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":63,"threads_started":1}
W20260926 14:56:30.069744 16167 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:30.066465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:43031 (local address 127.7.37.254:43643)
0926 14:56:30.066503 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:30.066505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:30.066564 (+ 59us) server_negotiation.cc:407] Connection header received
0926 14:56:30.066614 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:30.066615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:30.066621 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:30.066651 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:30.066820 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.067216 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.067809 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.067972 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.068014 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:30.069706 (+ 1692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.195:43031: BlockingRecv error: recv got EOF from 127.7.37.195:43031 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260926 14:56:30.069967 16429 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 26 14:56:30 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:30 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:30.849444 16435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:30.841256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.193:60365)
0926 14:56:30.841384 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:30.841395 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:30.841437 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:30.841555 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:30.841556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:30.841691 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:30.841735 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:30.841736 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.842326 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.842327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:30.842865 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:30.842867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.843185 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.843185 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:30.843220 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:30.847560 (+ 4340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:30.847565 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:30.849386 (+ 1821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
W20260926 14:56:30.849529 16436 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:30.841308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:60365 (local address 127.7.37.254:43643)
0926 14:56:30.841420 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:30.841422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:30.841430 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:30.841483 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:30.841484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:30.841492 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:30.841548 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:30.841783 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.842264 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.842939 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.843131 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.843182 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:30.849464 (+ 6282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.193:60365: BlockingRecv error: recv got EOF from 127.7.37.193:60365 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":53,"threads_started":1}
Sep 26 14:56:30 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:30 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:30.960364 16437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:30.956207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.194:54727)
0926 14:56:30.956325 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:30.956334 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:30.956386 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:30.956488 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:30.956488 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:30.956617 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:30.956659 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:30.956660 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.957351 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.957352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:30.957898 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:30.957900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.958246 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.958247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:30.958294 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:30.958702 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:30.958711 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:30.960285 (+ 1574us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
W20260926 14:56:30.960436 16436 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:30.956250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:54727 (local address 127.7.37.254:43643)
0926 14:56:30.956309 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:30.956311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:30.956386 (+ 75us) server_negotiation.cc:407] Connection header received
0926 14:56:30.956424 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:30.956425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:30.956437 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:30.956486 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:30.956706 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.957190 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.957973 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:30.958168 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:30.958228 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:30.960379 (+ 2151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.194:54727: BlockingRecv error: recv got EOF from 127.7.37.194:54727 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 26 14:56:31 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:31 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:31.074290 16438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:31.070251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.195:43841)
0926 14:56:31.070379 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:31.070388 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:31.070444 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:31.070578 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:31.070579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:31.070756 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:31.070810 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:31.070812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:31.071443 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:31.071443 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:31.072043 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:31.072045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:31.072352 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:31.072353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:31.072387 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:31.072744 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:31.072748 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:31.074213 (+ 1465us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":72,"threads_started":1}
W20260926 14:56:31.074342 16436 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:31.070308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:43841 (local address 127.7.37.254:43643)
0926 14:56:31.070366 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:31.070368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:31.070441 (+ 73us) server_negotiation.cc:407] Connection header received
0926 14:56:31.070494 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:31.070495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:31.070507 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:31.070567 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:31.070879 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:31.071380 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:31.072112 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:31.072300 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:31.072352 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:31.074295 (+ 1943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.195:43841: BlockingRecv error: recv got EOF from 127.7.37.195:43841 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 26 14:56:31 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:31 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:31.853857 16439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:31.849853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.193:55057)
0926 14:56:31.849970 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:31.849981 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:31.850025 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:31.850143 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:31.850144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:31.850288 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:31.850333 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:31.850334 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:31.850928 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:31.850929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:31.851448 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:31.851450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:31.851781 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:31.851781 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:31.851823 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:31.852194 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:31.852197 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:31.853793 (+ 1596us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":73,"threads_started":1}
W20260926 14:56:31.853940 16440 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:31.849923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:55057 (local address 127.7.37.254:43643)
0926 14:56:31.850039 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:31.850041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:31.850049 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:31.850067 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:31.850068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:31.850077 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:31.850133 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:31.850384 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:31.850867 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:31.851505 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:31.851688 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:31.851765 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:31.853877 (+ 2112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.193:55057: BlockingRecv error: recv got EOF from 127.7.37.193:55057 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
Sep 26 14:56:31 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:31 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:31.964874 16441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:31.960787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.194:41257)
0926 14:56:31.960919 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:31.960930 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:31.960992 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:31.961118 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:31.961119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:31.961276 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:31.961324 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:31.961326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:31.961915 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:31.961916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:31.962438 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:31.962440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:31.962793 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:31.962794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:31.962844 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:31.963231 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:31.963235 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:31.964810 (+ 1575us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
W20260926 14:56:31.964933 16440 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:31.960857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:41257 (local address 127.7.37.254:43643)
0926 14:56:31.960912 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:31.960914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:31.960983 (+ 69us) server_negotiation.cc:407] Connection header received
0926 14:56:31.961051 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:31.961051 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:31.961061 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:31.961104 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:31.961373 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:31.961854 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:31.962512 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:31.962702 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:31.962756 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:31.964888 (+ 2132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.194:41257: BlockingRecv error: recv got EOF from 127.7.37.194:41257 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 26 14:56:32 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:32 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:32.078676 16442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:32.074682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.195:42977)
0926 14:56:32.074787 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:32.074797 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:32.074847 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:32.074959 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:32.074960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:32.075114 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:32.075158 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:32.075159 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:32.075800 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:32.075801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:32.076341 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:32.076344 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:32.076672 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:32.076673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:32.076714 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:32.077104 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:32.077107 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:32.078605 (+ 1498us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":50,"threads_started":1}
W20260926 14:56:32.078748 16440 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:32.074802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:42977 (local address 127.7.37.254:43643)
0926 14:56:32.074843 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:32.074845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:32.074853 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:32.074892 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:32.074893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:32.074905 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:32.074949 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:32.075228 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:32.075708 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:32.076412 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:32.076602 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:32.076660 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:32.078693 (+ 2033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.195:42977: BlockingRecv error: recv got EOF from 127.7.37.195:42977 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 26 14:56:32 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:32 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:32.858393 16443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:32.854267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.193:42803)
0926 14:56:32.854395 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:32.854406 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:32.854448 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:32.854561 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:32.854561 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:32.854709 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:32.854754 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:32.854755 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:32.855351 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:32.855352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:32.855904 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:32.855906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:32.856324 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:32.856325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:32.856385 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:32.856751 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:32.856755 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:32.858340 (+ 1585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
W20260926 14:56:32.858510 16444 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:32.854332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:42803 (local address 127.7.37.254:43643)
0926 14:56:32.854426 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:32.854428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:32.854449 (+ 21us) server_negotiation.cc:407] Connection header received
0926 14:56:32.854495 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:32.854496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:32.854505 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:32.854545 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:32.854808 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:32.855295 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:32.855975 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:32.856257 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:32.856321 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:32.858436 (+ 2115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.193:42803: BlockingRecv error: recv got EOF from 127.7.37.193:42803 (error 108)
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":51,"threads_started":1}
Sep 26 14:56:32 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:32 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:32.969295 16445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:32.965283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.194:45817)
0926 14:56:32.965400 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:32.965412 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:32.965479 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:32.965605 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:32.965606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:32.965767 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:32.965809 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:32.965811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:32.966437 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:32.966438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:32.966965 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:32.966966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:32.967278 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:32.967279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:32.967333 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:32.967723 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:32.967728 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:32.969229 (+ 1501us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
W20260926 14:56:32.969377 16444 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:32.965352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:45817 (local address 127.7.37.254:43643)
0926 14:56:32.965422 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:32.965424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:32.965461 (+ 37us) server_negotiation.cc:407] Connection header received
0926 14:56:32.965535 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:32.965535 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:32.965547 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:32.965593 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:32.965875 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:32.966359 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:32.967033 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:32.967223 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:32.967275 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:32.969326 (+ 2051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.194:45817: BlockingRecv error: recv got EOF from 127.7.37.194:45817 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 26 14:56:33 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:33 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:33.083463 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:33.079040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.195:47267)
0926 14:56:33.079160 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:33.079172 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:33.079237 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:33.079382 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:33.079383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:33.079550 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:33.079596 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:33.079598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:33.080342 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:33.080343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:33.080874 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:33.080876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:33.081226 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:33.081227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:33.081284 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:33.081702 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:33.081706 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:33.083403 (+ 1697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":54,"threads_started":1}
W20260926 14:56:33.083544 16444 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:33.079113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:47267 (local address 127.7.37.254:43643)
0926 14:56:33.079167 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:33.079169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:33.079217 (+ 48us) server_negotiation.cc:407] Connection header received
0926 14:56:33.079297 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:33.079298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:33.079312 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:33.079374 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:33.079670 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:33.080250 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:33.080947 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:33.081153 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:33.081208 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:33.083485 (+ 2277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.195:47267: BlockingRecv error: recv got EOF from 127.7.37.195:47267 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 26 14:56:33 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:33 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:33.863071 16447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:33.858809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.193:49941)
0926 14:56:33.858926 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:33.858936 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:33.858983 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:33.859114 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:33.859115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:33.859271 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:33.859320 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:33.859322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:33.859956 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:33.859957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:33.860551 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:33.860553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:33.860849 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:33.860850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:33.860891 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:33.861327 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:33.861332 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:33.863018 (+ 1686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
W20260926 14:56:33.863164 16448 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:33.858865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:49941 (local address 127.7.37.254:43643)
0926 14:56:33.858964 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:33.858966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:33.858973 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:33.859036 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:33.859036 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:33.859046 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:33.859100 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:33.859373 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:33.859877 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:33.860610 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:33.860796 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:33.860842 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:33.863095 (+ 2253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.193:49941: BlockingRecv error: recv got EOF from 127.7.37.193:49941 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":47,"threads_started":1}
Sep 26 14:56:33 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:33 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:33.973628 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:33.969680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.194:40513)
0926 14:56:33.969801 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:33.969812 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:33.969869 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:33.969985 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:33.969986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:33.970167 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:33.970206 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:33.970208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:33.970801 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:33.970801 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:33.971335 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:33.971337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:33.971660 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:33.971660 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:33.971713 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:33.972111 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:33.972115 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:33.973580 (+ 1465us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":57,"threads_started":1}
W20260926 14:56:33.973726 16448 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:33.969744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:40513 (local address 127.7.37.254:43643)
0926 14:56:33.969802 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:33.969804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:33.969852 (+ 48us) server_negotiation.cc:407] Connection header received
0926 14:56:33.969916 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:33.969916 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:33.969928 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:33.969975 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:33.970268 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:33.970737 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:33.971401 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:33.971600 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:33.971664 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:33.973664 (+ 2000us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.194:40513: BlockingRecv error: recv got EOF from 127.7.37.194:40513 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 26 14:56:34 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:34 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:34.088166 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:34.083900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.195:39335)
0926 14:56:34.084031 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:34.084043 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:34.084110 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:34.084279 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:34.084280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:34.084447 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:34.084492 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:34.084493 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:34.085134 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:34.085135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:34.085660 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:34.085662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:34.085996 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:34.085997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:34.086055 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:34.086467 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:34.086472 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:34.088073 (+ 1601us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
W20260926 14:56:34.088213 16448 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:34.083948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:39335 (local address 127.7.37.254:43643)
0926 14:56:34.084038 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:34.084041 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:34.084090 (+ 49us) server_negotiation.cc:407] Connection header received
0926 14:56:34.084201 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:34.084202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:34.084215 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:34.084268 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:34.084561 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:34.085052 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:34.085738 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:34.085925 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:34.085978 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:34.088162 (+ 2184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.195:39335: BlockingRecv error: recv got EOF from 127.7.37.195:39335 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 26 14:56:34 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:34 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:34.867486 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:34.863466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.193:60409)
0926 14:56:34.863605 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:34.863616 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:34.863675 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:34.863802 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:34.863803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:34.863944 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:34.863990 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:34.863992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:34.864634 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:34.864635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:34.865154 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:34.865155 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:34.865435 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:34.865435 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:34.865476 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:34.865867 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:34.865870 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:34.867431 (+ 1561us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
W20260926 14:56:34.867586 16452 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:34.863538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:60409 (local address 127.7.37.254:43643)
0926 14:56:34.863665 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:34.863667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:34.863690 (+ 23us) server_negotiation.cc:407] Connection header received
0926 14:56:34.863710 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:34.863710 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:34.863720 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:34.863786 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:34.864078 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:34.864570 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:34.865215 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:34.865389 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:34.865430 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:34.867516 (+ 2086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.193:60409: BlockingRecv error: recv got EOF from 127.7.37.193:60409 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
Sep 26 14:56:34 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:34 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:34.977859 16453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:34.974004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.194:35401)
0926 14:56:34.974128 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:34.974138 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:34.974191 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:34.974302 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:34.974303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:34.974438 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:34.974482 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:34.974483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:34.975043 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:34.975044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:34.975547 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:34.975548 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:34.975835 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:34.975836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:34.975874 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:34.976235 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:34.976240 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:34.977805 (+ 1565us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":50,"threads_started":1}
W20260926 14:56:34.977942 16452 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:34.974094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:35401 (local address 127.7.37.254:43643)
0926 14:56:34.974142 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:34.974144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:34.974178 (+ 34us) server_negotiation.cc:407] Connection header received
0926 14:56:34.974237 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:34.974238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:34.974246 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:34.974287 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:34.974530 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:34.974993 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:34.975598 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:34.975786 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:34.975831 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:34.977882 (+ 2051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.194:35401: BlockingRecv error: recv got EOF from 127.7.37.194:35401 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Sep 26 14:56:35 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:35 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:35.092842 16454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:35.088601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.195:50839)
0926 14:56:35.088718 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:35.088730 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:35.088800 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:35.088911 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:35.088912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:35.089053 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:35.089099 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:35.089100 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:35.089675 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:35.089676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:35.090194 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:35.090195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:35.090598 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:35.090599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:35.090662 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:35.091105 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:35.091110 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:35.092768 (+ 1658us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":62,"threads_started":1}
W20260926 14:56:35.092904 16452 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:35.088696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:50839 (local address 127.7.37.254:43643)
0926 14:56:35.088756 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:35.088758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:35.088798 (+ 40us) server_negotiation.cc:407] Connection header received
0926 14:56:35.088850 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:35.088851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:35.088862 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:35.088898 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:35.089149 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:35.089623 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:35.090259 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:35.090523 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:35.090577 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:35.092853 (+ 2276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.195:50839: BlockingRecv error: recv got EOF from 127.7.37.195:50839 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 26 14:56:35 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:35 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:35.877255 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:35.867874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.193:50505)
0926 14:56:35.867994 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:35.868003 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:35.868051 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:35.868177 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:35.868178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:35.868320 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:35.868366 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:35.868367 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:35.868940 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:35.868941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:35.869466 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:35.869467 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:35.869734 (+ 267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:35.869735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:35.869769 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:35.875620 (+ 5851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:35.875625 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:35.877184 (+ 1559us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
W20260926 14:56:35.877339 16456 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:35.867940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:50505 (local address 127.7.37.254:43643)
0926 14:56:35.868040 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:35.868042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:35.868050 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:35.868089 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:35.868090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:35.868099 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:35.868149 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:35.868412 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:35.868890 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:35.869514 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:35.869693 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:35.869732 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:35.877280 (+ 7548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.193:50505: BlockingRecv error: recv got EOF from 127.7.37.193:50505 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":45,"threads_started":1}
Sep 26 14:56:35 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:35 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:35.982205 16457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:35.978239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.194:49459)
0926 14:56:35.978333 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:35.978345 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:35.978411 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:35.978536 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:35.978536 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:35.978678 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:35.978720 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:35.978722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:35.979317 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:35.979318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:35.979889 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:35.979891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:35.980204 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:35.980205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:35.980253 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:35.980612 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:35.980616 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:35.982137 (+ 1521us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":49,"threads_started":1}
W20260926 14:56:35.982306 16456 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:35.978309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:49459 (local address 127.7.37.254:43643)
0926 14:56:35.978368 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:35.978370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:35.978393 (+ 23us) server_negotiation.cc:407] Connection header received
0926 14:56:35.978474 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:35.978475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:35.978485 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:35.978523 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:35.978783 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:35.979253 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:35.979955 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:35.980142 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:35.980200 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:35.982239 (+ 2039us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.194:49459: BlockingRecv error: recv got EOF from 127.7.37.194:49459 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 26 14:56:36 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:36 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:36.097396 16458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:36.093271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.195:51929)
0926 14:56:36.093400 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:36.093411 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:36.093469 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:36.093592 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:36.093593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:36.093740 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:36.093784 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:36.093786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:36.094388 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:36.094390 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:36.094922 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:36.094924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:36.095230 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:36.095231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:36.095291 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:36.095696 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:36.095701 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:36.097326 (+ 1625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
W20260926 14:56:36.097463 16456 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:36.093326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:51929 (local address 127.7.37.254:43643)
0926 14:56:36.093373 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:36.093375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:36.093463 (+ 88us) server_negotiation.cc:407] Connection header received
0926 14:56:36.093528 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:36.093529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:36.093540 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:36.093581 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:36.093851 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:36.094323 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:36.094996 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:36.095175 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:36.095226 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:36.097411 (+ 2185us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.195:51929: BlockingRecv error: recv got EOF from 127.7.37.195:51929 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 26 14:56:36 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:36 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:36.881646 16459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:36.877712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.193:35511)
0926 14:56:36.877830 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:36.877839 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:36.877884 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:36.877998 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:36.877998 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:36.878138 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:36.878182 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:36.878184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:36.878769 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:36.878769 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:36.879287 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:36.879288 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:36.879568 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:36.879569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:36.879611 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:36.879998 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:36.880002 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:36.881585 (+ 1583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":50,"threads_started":1}
W20260926 14:56:36.881747 16460 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:36.877782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:35511 (local address 127.7.37.254:43643)
0926 14:56:36.877885 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:36.877887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:36.877916 (+ 29us) server_negotiation.cc:407] Connection header received
0926 14:56:36.877933 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:36.877934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:36.877943 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:36.877989 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:36.878232 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:36.878717 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:36.879338 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:36.879514 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:36.879556 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:36.881678 (+ 2122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.193:35511: BlockingRecv error: recv got EOF from 127.7.37.193:35511 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":50,"threads_started":1}
Sep 26 14:56:36 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:36 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:36.986534 16461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:36.982618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.194:46763)
0926 14:56:36.982740 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:36.982750 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:36.982805 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:36.982915 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:36.982915 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:36.983044 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:36.983087 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:36.983088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:36.983693 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:36.983693 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:36.984235 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:36.984236 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:36.984553 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:36.984553 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:36.984588 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:36.984975 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:36.984980 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:36.986482 (+ 1502us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
W20260926 14:56:36.986665 16460 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:36.982672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:46763 (local address 127.7.37.254:43643)
0926 14:56:36.982721 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:36.982723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:36.982808 (+ 85us) server_negotiation.cc:407] Connection header received
0926 14:56:36.982846 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:36.982846 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:36.982857 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:36.982905 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:36.983137 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:36.983627 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:36.984305 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:36.984495 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:36.984555 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:36.986602 (+ 2047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.194:46763: BlockingRecv error: recv got EOF from 127.7.37.194:46763 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 26 14:56:37 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:37 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:37.101945 16462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:37.097803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.195:41157)
0926 14:56:37.097928 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:37.097938 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:37.097989 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:37.098109 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:37.098109 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:37.098253 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:37.098298 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:37.098300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:37.098909 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:37.098910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:37.099449 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:37.099451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:37.099803 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:37.099804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:37.099855 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:37.100238 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:37.100242 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:37.101877 (+ 1635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
W20260926 14:56:37.101997 16460 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:37.097875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:41157 (local address 127.7.37.254:43643)
0926 14:56:37.097939 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:37.097941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:37.097986 (+ 45us) server_negotiation.cc:407] Connection header received
0926 14:56:37.098037 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:37.098038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:37.098049 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:37.098096 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:37.098345 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:37.098830 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:37.099516 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:37.099697 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:37.099762 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:37.101951 (+ 2189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.195:41157: BlockingRecv error: recv got EOF from 127.7.37.195:41157 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 26 14:56:37 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:37 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:37.886098 16463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:37.882030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.193:49467)
0926 14:56:37.882144 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:37.882155 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:37.882201 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:37.882337 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:37.882337 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:37.882476 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:37.882522 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:37.882523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:37.883109 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:37.883110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:37.883642 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:37.883644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:37.883963 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:37.883964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:37.884004 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:37.884400 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:37.884405 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:37.886027 (+ 1622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":52,"threads_started":1}
W20260926 14:56:37.886176 16464 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:37.882120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:49467 (local address 127.7.37.254:43643)
0926 14:56:37.882232 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:37.882234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:37.882240 (+ 6us) server_negotiation.cc:407] Connection header received
0926 14:56:37.882258 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:37.882258 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:37.882268 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:37.882319 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:37.882572 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:37.883054 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:37.883711 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:37.883911 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:37.883964 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:37.886115 (+ 2151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.193:49467: BlockingRecv error: recv got EOF from 127.7.37.193:49467 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":58,"threads_started":1}
Sep 26 14:56:37 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:37 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:37.991325 16465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:37.986894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.194:55753)
0926 14:56:37.987008 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:37.987020 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:37.987094 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:37.987232 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:37.987233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:37.987394 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:37.987439 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:37.987441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:37.988281 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:37.988282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:37.988827 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:37.988830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:37.989184 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:37.989185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:37.989239 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:37.989628 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:37.989633 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:37.991262 (+ 1629us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":54,"threads_started":1}
W20260926 14:56:37.991415 16464 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:37.986955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:55753 (local address 127.7.37.254:43643)
0926 14:56:37.987009 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:37.987011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:37.987066 (+ 55us) server_negotiation.cc:407] Connection header received
0926 14:56:37.987157 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:37.987158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:37.987171 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:37.987217 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:37.987512 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:37.988206 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:37.988906 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:37.989114 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:37.989167 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:37.991355 (+ 2188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.194:55753: BlockingRecv error: recv got EOF from 127.7.37.194:55753 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 26 14:56:38 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:38 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:38.106397 16466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:38.102346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.195:55843)
0926 14:56:38.102465 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:38.102475 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:38.102525 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:38.102628 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:38.102628 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:38.102780 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:38.102834 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:38.102836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:38.103565 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:38.103566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:38.104126 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:38.104128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:38.104442 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:38.104442 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:38.104494 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:38.104854 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:38.104858 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:38.106326 (+ 1468us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":65,"threads_started":1}
W20260926 14:56:38.106469 16464 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:38.102412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:55843 (local address 127.7.37.254:43643)
0926 14:56:38.102467 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:38.102469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:38.102513 (+ 44us) server_negotiation.cc:407] Connection header received
0926 14:56:38.102562 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:38.102562 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:38.102572 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:38.102617 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:38.102903 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:38.103495 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:38.104199 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:38.104387 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:38.104437 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:38.106417 (+ 1980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.195:55843: BlockingRecv error: recv got EOF from 127.7.37.195:55843 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 26 14:56:38 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:38 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:38.890762 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:38.886497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.193:37725)
0926 14:56:38.886605 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:38.886615 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:38.886664 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:38.886784 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:38.886785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:38.886933 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:38.886979 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:38.886981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:38.887574 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:38.887574 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:38.888153 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:38.888156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:38.888463 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:38.888463 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:38.888498 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:38.888920 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:38.888924 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:38.890666 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":58,"threads_started":1}
W20260926 14:56:38.890830 16468 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:38.886558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:37725 (local address 127.7.37.254:43643)
0926 14:56:38.886662 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:38.886663 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:38.886671 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:38.886714 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:38.886714 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:38.886724 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:38.886774 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:38.887031 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:38.887511 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:38.888222 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:38.888410 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:38.888460 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:38.890757 (+ 2297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.193:37725: BlockingRecv error: recv got EOF from 127.7.37.193:37725 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":49,"threads_started":1}
Sep 26 14:56:38 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:38 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:38.995558 16469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:38.991691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.194:43505)
0926 14:56:38.991818 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:38.991828 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:38.991885 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:38.991994 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:38.991994 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:38.992132 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:38.992179 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:38.992180 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:38.992751 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:38.992751 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:38.993254 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:38.993255 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:38.993542 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:38.993542 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:38.993576 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:38.993964 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:38.993968 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:38.995504 (+ 1536us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":76,"threads_started":1}
W20260926 14:56:38.995653 16468 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:38.991730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:43505 (local address 127.7.37.254:43643)
0926 14:56:38.991786 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:38.991788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:38.991881 (+ 93us) server_negotiation.cc:407] Connection header received
0926 14:56:38.991935 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:38.991935 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:38.991945 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:38.991980 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:38.992229 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:38.992700 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:38.993311 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:38.993492 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:38.993538 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:38.995587 (+ 2049us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.194:43505: BlockingRecv error: recv got EOF from 127.7.37.194:43505 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 26 14:56:39 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:39 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:39.111174 16470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:39.106810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.195:40123)
0926 14:56:39.106929 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:39.106940 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:39.106995 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:39.107091 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:39.107092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:39.107235 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:39.107290 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:39.107292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:39.108085 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:39.108086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:39.108622 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:39.108624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:39.108994 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:39.108995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:39.109040 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:39.109480 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:39.109486 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:39.111112 (+ 1626us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
W20260926 14:56:39.111255 16468 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:39.106892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:40123 (local address 127.7.37.254:43643)
0926 14:56:39.106936 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:39.106938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:39.106981 (+ 43us) server_negotiation.cc:407] Connection header received
0926 14:56:39.107033 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:39.107034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:39.107044 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:39.107085 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:39.107356 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:39.108013 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:39.108697 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:39.108936 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:39.108984 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:39.111198 (+ 2214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.195:40123: BlockingRecv error: recv got EOF from 127.7.37.195:40123 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 26 14:56:39 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:39 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:39.895327 16471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:39.891192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.193:43157)
0926 14:56:39.891310 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:39.891321 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:39.891366 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:39.891489 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:39.891489 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:39.891643 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:39.891688 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:39.891690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:39.892312 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:39.892312 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:39.893028 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:39.893031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:39.893334 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:39.893334 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:39.893368 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:39.893761 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:39.893764 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:39.895259 (+ 1495us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":54,"threads_started":1}
W20260926 14:56:39.895424 16472 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:39.891248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:43157 (local address 127.7.37.254:43643)
0926 14:56:39.891348 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:39.891350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:39.891359 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:56:39.891413 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:39.891413 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:39.891423 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:39.891480 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:39.891763 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:39.892254 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:39.893087 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:39.893283 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:39.893331 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:39.895354 (+ 2023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.193:43157: BlockingRecv error: recv got EOF from 127.7.37.193:43157 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":50,"threads_started":1}
Sep 26 14:56:39 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:39 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:40.000203 16473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:39.995984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.194:49323)
0926 14:56:39.996116 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:39.996127 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:39.996174 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:39.996276 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:39.996277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:39.996426 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:39.996470 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:39.996471 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:39.997039 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:39.997040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:39.997543 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:39.997544 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:39.997858 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:39.997858 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:39.997891 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:39.998374 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:39.998379 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:40.000143 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
W20260926 14:56:40.000355 16472 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:39.996076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:49323 (local address 127.7.37.254:43643)
0926 14:56:39.996171 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:39.996173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:39.996182 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:56:39.996207 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:39.996208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:39.996217 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:39.996266 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:39.996521 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:39.996990 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:39.997613 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:39.997804 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:39.997856 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:40.000289 (+ 2433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.194:49323: BlockingRecv error: recv got EOF from 127.7.37.194:49323 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 26 14:56:40 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:40 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:40.115859 16474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:40.111556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.195:49249)
0926 14:56:40.111657 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:40.111667 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:40.111715 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:40.111844 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:40.111845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:40.112001 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:40.112044 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:40.112045 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:40.112659 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:40.112660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:40.113173 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:40.113175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:40.113486 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:40.113487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:40.113541 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:40.113942 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:40.113946 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:40.115788 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":47,"threads_started":1}
W20260926 14:56:40.115958 16472 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:40.111668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:49249 (local address 127.7.37.254:43643)
0926 14:56:40.111727 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:40.111729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:40.111736 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:40.111775 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:40.111776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:40.111789 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:40.111834 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:40.112104 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:40.112589 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:40.113246 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:40.113426 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:40.113476 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:40.115889 (+ 2413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.195:49249: BlockingRecv error: recv got EOF from 127.7.37.195:49249 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 26 14:56:40 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:40 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:40.904194 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:40.895730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.193:39649)
0926 14:56:40.895862 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:40.895871 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:40.895916 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:40.896030 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:40.896031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:40.896206 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:40.896270 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:40.896271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:40.896949 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:40.896950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:40.897620 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:40.897623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:40.897926 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:40.897926 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:40.897971 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:40.902539 (+ 4568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:40.902544 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:40.904130 (+ 1586us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
W20260926 14:56:40.904263 16476 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:40.895792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:39649 (local address 127.7.37.254:43643)
0926 14:56:40.895902 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:40.895904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:40.895912 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:40.895955 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:40.895956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:40.895966 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:40.896020 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:40.896369 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:40.896873 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:40.897682 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:40.897875 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:40.897923 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:40.904207 (+ 6284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.193:39649: BlockingRecv error: recv got EOF from 127.7.37.193:39649 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":51,"threads_started":1}
Sep 26 14:56:40 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:41 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:41.004761 16477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:41.000588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.194:40427)
0926 14:56:41.000711 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:41.000721 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:41.000777 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:41.000884 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:41.000885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:41.001031 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:41.001076 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:41.001077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:41.001677 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:41.001678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:41.002237 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:41.002239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:41.002527 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:41.002528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:41.002583 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:41.003024 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:41.003028 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:41.004697 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":57,"threads_started":1}
W20260926 14:56:41.004827 16476 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:41.000645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:40427 (local address 127.7.37.254:43643)
0926 14:56:41.000707 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:41.000710 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:41.000763 (+ 53us) server_negotiation.cc:407] Connection header received
0926 14:56:41.000825 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:41.000826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:41.000835 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:41.000871 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:41.001135 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:41.001594 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:41.002299 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:41.002472 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:41.002517 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:41.004778 (+ 2261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.194:40427: BlockingRecv error: recv got EOF from 127.7.37.194:40427 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Sep 26 14:56:41 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:41 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:41.120743 16478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:41.116312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.195:51119)
0926 14:56:41.116439 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:41.116462 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:41.116527 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:41.116646 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:41.116647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:41.116816 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:41.116859 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:41.116860 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:41.117469 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:41.117470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:41.117998 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:41.118000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:41.118328 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:41.118329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:41.118392 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:41.118862 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:41.118867 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:41.120657 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":57,"threads_started":1}
W20260926 14:56:41.120800 16476 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:41.116383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:51119 (local address 127.7.37.254:43643)
0926 14:56:41.116427 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:41.116429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:41.116519 (+ 90us) server_negotiation.cc:407] Connection header received
0926 14:56:41.116583 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:41.116584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:41.116596 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:41.116641 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:41.116923 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:41.117403 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:41.118069 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:41.118272 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:41.118323 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:41.120746 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.195:51119: BlockingRecv error: recv got EOF from 127.7.37.195:51119 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 26 14:56:41 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:41 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:41.908619 16479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:41.904557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.193:50447)
0926 14:56:41.904656 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:41.904673 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:41.904720 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:41.904828 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:41.904829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:41.904973 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:41.905015 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:41.905016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:41.905634 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:41.905635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:41.906181 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:41.906182 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:41.906503 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:41.906504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:41.906555 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:41.906924 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:41.906928 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:41.908559 (+ 1631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":58,"threads_started":1}
W20260926 14:56:41.908711 16480 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:41.904612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:50447 (local address 127.7.37.254:43643)
0926 14:56:41.904710 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:41.904713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:41.904721 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:41.904768 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:41.904769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:41.904779 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:41.904818 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:41.905067 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:41.905564 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:41.906244 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:41.906429 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:41.906475 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:41.908644 (+ 2169us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.193:50447: BlockingRecv error: recv got EOF from 127.7.37.193:50447 (error 108)
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":41,"threads_started":1}
Sep 26 14:56:42 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:42 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:42.009341 16481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:42.005184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.194:56411)
0926 14:56:42.005300 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:42.005310 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:42.005368 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:42.005469 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:42.005469 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:42.005610 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:42.005658 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:42.005660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:42.006344 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:42.006345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:42.006865 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:42.006866 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:42.007194 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:42.007195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:42.007233 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:42.007647 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:42.007651 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:42.009269 (+ 1618us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
W20260926 14:56:42.009431 16480 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:42.005244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:56411 (local address 127.7.37.254:43643)
0926 14:56:42.005319 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:42.005321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:42.005365 (+ 44us) server_negotiation.cc:407] Connection header received
0926 14:56:42.005417 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:42.005418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:42.005427 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:42.005458 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:42.005704 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:42.006269 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:42.006935 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:42.007124 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:42.007187 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:42.009362 (+ 2175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.194:56411: BlockingRecv error: recv got EOF from 127.7.37.194:56411 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 26 14:56:42 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:42 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:42.125237 16482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:42.121194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.195:43361)
0926 14:56:42.121319 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:42.121329 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:42.121387 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:42.121491 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:42.121491 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:42.121647 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:42.121689 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:42.121691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:42.122306 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:42.122307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:42.122898 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:42.122901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:42.123185 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:42.123186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:42.123239 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:42.123610 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:42.123614 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:42.125160 (+ 1546us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":56,"threads_started":1}
W20260926 14:56:42.125283 16480 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:42.121248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:43361 (local address 127.7.37.254:43643)
0926 14:56:42.121301 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:42.121303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:42.121379 (+ 76us) server_negotiation.cc:407] Connection header received
0926 14:56:42.121426 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:42.121426 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:42.121437 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:42.121479 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:42.121733 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:42.122220 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:42.122957 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:42.123133 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:42.123178 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:42.125235 (+ 2057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.195:43361: BlockingRecv error: recv got EOF from 127.7.37.195:43361 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Sep 26 14:56:42 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:42 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:42.913285 16483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:42.908998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.193:38937)
0926 14:56:42.909147 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:42.909158 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:42.909217 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:42.909331 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:42.909331 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:42.909474 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:42.909520 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:42.909521 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:42.910182 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:42.910183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:42.910721 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:42.910722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:42.911074 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:42.911075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:42.911141 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:42.911635 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:42.911639 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:42.913219 (+ 1580us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":63,"threads_started":1}
W20260926 14:56:42.913383 16484 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:42.909048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:38937 (local address 127.7.37.254:43643)
0926 14:56:42.909150 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:42.909152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:42.909204 (+ 52us) server_negotiation.cc:407] Connection header received
0926 14:56:42.909257 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:42.909257 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:42.909267 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:42.909332 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:42.909573 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:42.910108 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:42.910816 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:42.911012 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:42.911069 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:42.913316 (+ 2247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.193:38937: BlockingRecv error: recv got EOF from 127.7.37.193:38937 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":54,"threads_started":1}
Sep 26 14:56:43 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:43 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:43.013914 16485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:43.009743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.194:45715)
0926 14:56:43.009875 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:43.009886 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:43.009941 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:43.010055 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:43.010056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:43.010218 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:43.010266 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:43.010267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:43.010873 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:43.010873 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:43.011409 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:43.011411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:43.011693 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:43.011694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:43.011735 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:43.012166 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:43.012171 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:43.013857 (+ 1686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":58,"threads_started":1}
W20260926 14:56:43.013994 16484 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:43.009864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:45715 (local address 127.7.37.254:43643)
0926 14:56:43.009915 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:43.009917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:43.009925 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:43.009990 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:43.009990 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:43.010001 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:43.010044 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:43.010318 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:43.010814 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:43.011468 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:43.011640 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:43.011682 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:43.013937 (+ 2255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.194:45715: BlockingRecv error: recv got EOF from 127.7.37.194:45715 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 26 14:56:43 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:43 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:43.129808 16486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:43.125620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.195:41699)
0926 14:56:43.125747 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:43.125756 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:43.125813 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:43.125916 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:43.125916 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:43.126063 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:43.126108 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:43.126110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:43.126756 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:43.126756 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:43.127281 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:43.127283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:43.127597 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:43.127598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:43.127640 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:43.128063 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:43.128068 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:43.129750 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
W20260926 14:56:43.129884 16484 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:43.125661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:41699 (local address 127.7.37.254:43643)
0926 14:56:43.125711 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:43.125714 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:43.125808 (+ 94us) server_negotiation.cc:407] Connection header received
0926 14:56:43.125850 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:43.125850 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:43.125861 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:43.125905 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:43.126193 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:43.126684 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:43.127346 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:43.127541 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:43.127591 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:43.129828 (+ 2237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.195:41699: BlockingRecv error: recv got EOF from 127.7.37.195:41699 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Sep 26 14:56:43 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:43 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:43.917814 16487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:43.913684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.193:36977)
0926 14:56:43.913795 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:43.913805 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:43.913850 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:43.913966 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:43.913967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:43.914134 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:43.914178 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:43.914180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:43.914770 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:43.914770 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:43.915302 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:43.915303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:43.915648 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:43.915649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:43.915701 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:43.916151 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:43.916155 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:43.917740 (+ 1585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
W20260926 14:56:43.917896 16488 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:43.913727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:36977 (local address 127.7.37.254:43643)
0926 14:56:43.913839 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:43.913841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:43.913850 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:56:43.913886 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:43.913886 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:43.913896 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:43.913949 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:43.914231 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:43.914712 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:43.915370 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:43.915563 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:43.915614 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:43.917835 (+ 2221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.193:36977: BlockingRecv error: recv got EOF from 127.7.37.193:36977 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":53,"threads_started":1}
Sep 26 14:56:44 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:44 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:44.018335 16489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:44.014319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.194:48629)
0926 14:56:44.014424 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:44.014436 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:44.014495 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:44.014599 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:44.014600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:44.014741 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:44.014784 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:44.014785 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:44.015359 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:44.015360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:44.015906 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:44.015908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:44.016231 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:44.016231 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:44.016283 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:44.016716 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:44.016720 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:44.018268 (+ 1548us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":50,"threads_started":1}
W20260926 14:56:44.018419 16488 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:44.014406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:48629 (local address 127.7.37.254:43643)
0926 14:56:44.014457 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:44.014458 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:44.014480 (+ 22us) server_negotiation.cc:407] Connection header received
0926 14:56:44.014534 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:44.014534 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:44.014544 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:44.014586 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:44.014831 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:44.015301 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:44.015965 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:44.016154 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:44.016206 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:44.018362 (+ 2156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.194:48629: BlockingRecv error: recv got EOF from 127.7.37.194:48629 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 26 14:56:44 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:44 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.195@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:44.134394 16490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:44.130214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.195:45999)
0926 14:56:44.130348 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:44.130359 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:44.130424 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:44.130556 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:44.130557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:44.130716 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:44.130776 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:44.130777 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:44.131395 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:44.131396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:44.132007 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:44.132009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:44.132312 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:44.132312 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:44.132347 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:44.132735 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:44.132739 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:44.134339 (+ 1600us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":55,"threads_started":1}
W20260926 14:56:44.134572 16488 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:44.130304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:45999 (local address 127.7.37.254:43643)
0926 14:56:44.130355 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:44.130356 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:44.130415 (+ 59us) server_negotiation.cc:407] Connection header received
0926 14:56:44.130489 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:44.130489 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:44.130500 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:44.130536 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:44.130838 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:44.131314 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:44.132079 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:44.132260 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:44.132308 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:44.134511 (+ 2203us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.195:45999: BlockingRecv error: recv got EOF from 127.7.37.195:45999 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 26 14:56:44 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:44 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.193@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:44.922757 16491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:44.918251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.193:58781)
0926 14:56:44.918351 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:44.918360 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:44.918408 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:44.918505 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:44.918505 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:44.918688 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:44.918744 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:44.918746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:44.919466 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:44.919467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:44.920144 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:44.920147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:44.920469 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:44.920469 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:44.920505 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:44.920946 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:44.920951 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:44.922683 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":59,"threads_started":1}
W20260926 14:56:44.922833 16492 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:44.918313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:58781 (local address 127.7.37.254:43643)
0926 14:56:44.918407 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:44.918408 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:44.918416 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:44.918440 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:44.918440 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:44.918449 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:44.918502 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:44.918798 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:44.919401 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:44.920218 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:44.920413 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:44.920467 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:44.922773 (+ 2306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.193:58781: BlockingRecv error: recv got EOF from 127.7.37.193:58781 (error 108)
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":43,"threads_started":1}
Sep 26 14:56:45 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 14:56:45 dist-test-slave-f9xb krb5kdc[15433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.37.194@KRBTEST.COM for oryx/127.7.37.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 14:56:45.023172 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:45.018845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43643 (local address 127.7.37.194:55233)
0926 14:56:45.018970 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:45.018981 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:45.019040 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:45.019156 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:45.019157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:45.019317 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:45.019361 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:45.019363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:45.020109 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:45.020110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:45.020637 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:45.020639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:45.020964 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:45.020965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:45.021016 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:45.021413 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:45.021417 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:45.023041 (+ 1624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.37.254:43643: Server oryx/127.7.37.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
W20260926 14:56:45.023201 16492 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:45.018899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:55233 (local address 127.7.37.254:43643)
0926 14:56:45.018959 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:45.018961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:45.019024 (+ 63us) server_negotiation.cc:407] Connection header received
0926 14:56:45.019089 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:45.019089 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:45.019100 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:45.019143 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:45.019425 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:45.020039 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:45.020710 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:45.020910 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:45.020962 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:45.023150 (+ 2188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.37.194:55233: BlockingRecv error: recv got EOF from 127.7.37.194:55233 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 14:56:45.067449 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 16017
I20260926 14:56:45.071197 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 16168
I20260926 14:56:45.074400 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 16300
I20260926 14:56:45.078318 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 15946
2026-09-26T14:56:45Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17224 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-26T14:56:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:45Z Disabled control of system clock
I20260926 14:56:45.102146 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:44823
--webserver_interface=127.7.37.254
--webserver_port=0
--builtin_ntp_servers=127.7.37.212:40619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:44823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 14:56:45.177664 16497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:45.177836 16497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:45.177855 16497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:45.178930 16497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:45.178972 16497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:45.178984 16497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:45.178996 16497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:45.179013 16497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:56:45.180563 16497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:40619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:44823
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:44823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:45.180770 16497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:45.181022 16497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:45.185659 16503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:45.185708 16505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:45.185850 16497 server_base.cc:1034] running on GCE node
W20260926 14:56:45.185873 16502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:45.186102 16497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:45.186376 16497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:45.187526 16497 hybrid_clock.cc:648] HybridClock initialized: now 1790434605187516 us; error 32 us; skew 500 ppm
I20260926 14:56:45.188709 16497 webserver.cc:460] Webserver started at http://127.7.37.254:38641/ using document root <none> and password file <none>
I20260926 14:56:45.188926 16497 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:45.188994 16497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:45.189158 16497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:45.190109 16497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "e9c3abd824574837ae921b0e9b098420"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "3fde5dfc098cd6acd0f0c92d37cf7964"
server_key_iv: "5e5ce9ec59a06069be2d8d0836fccff9"
server_key_version: "encryptionkey@0"
I20260926 14:56:45.190428 16497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "e9c3abd824574837ae921b0e9b098420"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "3fde5dfc098cd6acd0f0c92d37cf7964"
server_key_iv: "5e5ce9ec59a06069be2d8d0836fccff9"
server_key_version: "encryptionkey@0"
I20260926 14:56:45.191624 16497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 14:56:45.192358 16511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:45.192509 16497 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:45.192588 16497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "e9c3abd824574837ae921b0e9b098420"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "3fde5dfc098cd6acd0f0c92d37cf7964"
server_key_iv: "5e5ce9ec59a06069be2d8d0836fccff9"
server_key_version: "encryptionkey@0"
I20260926 14:56:45.192657 16497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:45.198999 16497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:45.199231 16497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:45.199352 16497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:45.203015 16497 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:44823
I20260926 14:56:45.203033 16563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:44823 every 8 connection(s)
I20260926 14:56:45.203315 16497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:56:45.203923 16564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:45.205440 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 16497
I20260926 14:56:45.205521 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:56:45.205694 7319 external_mini_cluster.cc:1428] Setting key 15f477d623a6fc86fadae3071de5534e
I20260926 14:56:45.206267 16564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420: Bootstrap starting.
I20260926 14:56:45.206888 16564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:45.207206 16564 log.cc:824] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:45.207893 16564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420: No bootstrap required, opened a new log
I20260926 14:56:45.209846 16564 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c3abd824574837ae921b0e9b098420" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 44823 } }
I20260926 14:56:45.209997 16564 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:45.210031 16564 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9c3abd824574837ae921b0e9b098420, State: Initialized, Role: FOLLOWER
I20260926 14:56:45.210170 16564 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c3abd824574837ae921b0e9b098420" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 44823 } }
I20260926 14:56:45.210263 16564 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:56:45.210302 16564 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:56:45.210346 16564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:45.210971 16564 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c3abd824574837ae921b0e9b098420" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 44823 } }
I20260926 14:56:45.211077 16564 leader_election.cc:302] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9c3abd824574837ae921b0e9b098420; no voters:
I20260926 14:56:45.211113 16567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:45.206093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45346 (local address 127.7.37.254:44823)
0926 14:56:45.206269 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:45.206277 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:45.206305 (+ 28us) server_negotiation.cc:407] Connection header received
0926 14:56:45.206803 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:45.206811 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:45.206884 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:45.207032 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:45.207215 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:45.208444 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:45.209168 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:45.209782 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:45.209904 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:45.210480 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:45.210484 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:45.210484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:45.210672 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:45.210709 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:45.210723 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:45.210801 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:45.210859 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 14:56:45.210931 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
I20260926 14:56:45.211203 16564 leader_election.cc:288] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:56:45.211261 16569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:45.211427 16569 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420 [term 1 LEADER]: Becoming Leader. State: Replica: e9c3abd824574837ae921b0e9b098420, State: Running, Role: LEADER
I20260926 14:56:45.211508 16569 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c3abd824574837ae921b0e9b098420" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 44823 } }
I20260926 14:56:45.211678 16564 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:45.211951 16571 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9c3abd824574837ae921b0e9b098420. Latest consensus state: current_term: 1 leader_uuid: "e9c3abd824574837ae921b0e9b098420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c3abd824574837ae921b0e9b098420" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 44823 } } }
I20260926 14:56:45.212045 16571 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:45.211946 16570 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9c3abd824574837ae921b0e9b098420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c3abd824574837ae921b0e9b098420" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 44823 } } }
I20260926 14:56:45.212189 16570 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:45.212518 16575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:56:45.212935 16575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:56:45.214139 16575 catalog_manager.cc:1347] Generated new cluster ID: 70ff06f5bbe645b780c100cc44a6b563
I20260926 14:56:45.214191 16575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:56:45.234411 16575 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:56:45.235085 16575 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:45.239420 16575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9c3abd824574837ae921b0e9b098420: Generated new TSK 0
I20260926 14:56:45.239570 16575 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:56:45.244599 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:44823
--builtin_ntp_servers=127.7.37.212:40619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 14:56:45.319079 16588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:45.319257 16588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:45.319284 16588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:45.320396 16588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:45.320432 16588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:45.320462 16588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:56:45.321695 16588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:40619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:44823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:45.321897 16588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:45.322136 16588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:45.327401 16596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:45.327423 16594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:45.327414 16593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:45.327559 16588 server_base.cc:1034] running on GCE node
I20260926 14:56:45.327785 16588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:45.328006 16588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:45.329151 16588 hybrid_clock.cc:648] HybridClock initialized: now 1790434605329130 us; error 37 us; skew 500 ppm
I20260926 14:56:45.330157 16588 webserver.cc:460] Webserver started at http://127.7.37.193:34145/ using document root <none> and password file <none>
I20260926 14:56:45.330377 16588 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:45.330422 16588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:45.330539 16588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:45.331485 16588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "762bfff8d48740a78792743dd863d624"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "a95d8fc0ae7f4548ab8329f23d0caa26"
server_key_iv: "5beb7b810e7878a2f090b86950f559ed"
server_key_version: "encryptionkey@0"
I20260926 14:56:45.331838 16588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "762bfff8d48740a78792743dd863d624"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "a95d8fc0ae7f4548ab8329f23d0caa26"
server_key_iv: "5beb7b810e7878a2f090b86950f559ed"
server_key_version: "encryptionkey@0"
I20260926 14:56:45.332930 16588 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:45.333518 16602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:45.333671 16588 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:45.333743 16588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "762bfff8d48740a78792743dd863d624"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "a95d8fc0ae7f4548ab8329f23d0caa26"
server_key_iv: "5beb7b810e7878a2f090b86950f559ed"
server_key_version: "encryptionkey@0"
I20260926 14:56:45.333806 16588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:45.355607 16588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:45.355865 16588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:45.355979 16588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:45.356143 16588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:45.356420 16588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:45.356451 16588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:45.356477 16588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:45.356500 16588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:45.361490 16588 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:39421
I20260926 14:56:45.361526 16715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:39421 every 8 connection(s)
I20260926 14:56:45.361792 16588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:56:45.366109 16567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:45.362445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:44949 (local address 127.7.37.254:44823)
0926 14:56:45.362494 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:45.362496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:45.362626 (+ 130us) server_negotiation.cc:407] Connection header received
0926 14:56:45.363030 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:45.363031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:45.363042 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:45.363094 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:45.363623 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:45.364185 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:45.365088 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:45.365272 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:45.365328 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:45.365902 (+ 574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:45.365903 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:45.365903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:45.365906 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:45.365920 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:45.365923 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:45.365956 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:45.366012 (+ 56us) server_negotiation.cc:299] Negotiation successful
0926 14:56:45.366036 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 14:56:45.366253 16718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:45.362361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:44823 (local address 127.7.37.193:44949)
0926 14:56:45.362521 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:45.362555 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:45.362960 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:45.363138 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:45.363141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:45.363148 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:45.363564 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:45.363567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:45.364255 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:45.364255 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:45.364989 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:45.364991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:45.365333 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:45.365333 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:45.365423 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:45.365797 (+ 374us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:56:45.365798 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:45.365820 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:56:45.365838 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:45.365962 (+ 124us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:45.365964 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:45.365966 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:56:45.366005 (+ 39us) client_negotiation.cc:240] Negotiation successful
0926 14:56:45.366070 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260926 14:56:45.366673 16716 heartbeater.cc:347] Connected to a master server at 127.7.37.254:44823
I20260926 14:56:45.366748 16716 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:45.366868 16716 heartbeater.cc:511] Master 127.7.37.254:44823 requested a full tablet report, sending...
I20260926 14:56:45.367295 16528 ts_manager.cc:194] Registered new tserver with Master: 762bfff8d48740a78792743dd863d624 (127.7.37.193:39421)
I20260926 14:56:45.367926 16528 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.37.193:44949
I20260926 14:56:45.368571 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 16588
I20260926 14:56:45.368642 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:56:45.368808 7319 external_mini_cluster.cc:1428] Setting key 8377a5ea84556f6281a903d81726800c
I20260926 14:56:45.369571 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:44823
--builtin_ntp_servers=127.7.37.212:40619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 14:56:45.447850 16719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:45.448014 16719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:45.448040 16719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:45.449117 16719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:45.449152 16719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:45.449183 16719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:56:45.450404 16719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:40619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:44823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:45.450600 16719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:45.450842 16719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:45.455117 16724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:45.455099 16727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:45.455190 16725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:45.455255 16719 server_base.cc:1034] running on GCE node
I20260926 14:56:45.455490 16719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:45.455693 16719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:45.456844 16719 hybrid_clock.cc:648] HybridClock initialized: now 1790434605456846 us; error 37 us; skew 500 ppm
I20260926 14:56:45.457947 16719 webserver.cc:460] Webserver started at http://127.7.37.194:44721/ using document root <none> and password file <none>
I20260926 14:56:45.458185 16719 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:45.458233 16719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:45.458344 16719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:45.459244 16719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "14e860c54546408da92e01c0af7d3916"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "b7b7e08a20b3f434f9b614eef07c4b91"
server_key_iv: "4419f5d595fbc99369c00d65012cc38e"
server_key_version: "encryptionkey@0"
I20260926 14:56:45.459530 16719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "14e860c54546408da92e01c0af7d3916"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "b7b7e08a20b3f434f9b614eef07c4b91"
server_key_iv: "4419f5d595fbc99369c00d65012cc38e"
server_key_version: "encryptionkey@0"
I20260926 14:56:45.460680 16719 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 14:56:45.461318 16733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:45.461472 16719 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:56:45.461535 16719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "14e860c54546408da92e01c0af7d3916"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "b7b7e08a20b3f434f9b614eef07c4b91"
server_key_iv: "4419f5d595fbc99369c00d65012cc38e"
server_key_version: "encryptionkey@0"
I20260926 14:56:45.461597 16719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:45.486038 16719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:45.486260 16719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:45.486366 16719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:45.486531 16719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:45.486791 16719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:45.486821 16719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:45.486846 16719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:45.486869 16719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:45.492041 16719 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:36107
I20260926 14:56:45.492089 16846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:36107 every 8 connection(s)
I20260926 14:56:45.492345 16719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:56:45.493469 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 16719
I20260926 14:56:45.493544 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:56:45.493726 7319 external_mini_cluster.cc:1428] Setting key 9d9dcaa00a99de1ed39c3ec4da5661bb
I20260926 14:56:45.494521 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:44823
--builtin_ntp_servers=127.7.37.212:40619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 14:56:45.497182 16567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:45.492923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:56813 (local address 127.7.37.254:44823)
0926 14:56:45.492968 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:45.492970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:45.493155 (+ 185us) server_negotiation.cc:407] Connection header received
0926 14:56:45.493587 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:45.493588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:45.493599 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:45.493667 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:45.494352 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:45.494910 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:45.496007 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:45.496211 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:45.496262 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:45.496984 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:45.496985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:45.496985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:45.496989 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:45.497002 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:45.497006 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:45.497041 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:45.497109 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 14:56:45.497128 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 14:56:45.497392 16849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:45.492889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:44823 (local address 127.7.37.194:56813)
0926 14:56:45.493050 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:45.493085 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:45.493496 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:45.493737 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:45.493740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:45.493748 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:45.494278 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:45.494283 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:45.494987 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:45.494988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:45.495935 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:45.495938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:45.496271 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:45.496272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:45.496410 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:45.496844 (+ 434us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:56:45.496844 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:45.496877 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:56:45.496900 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:45.497050 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:45.497053 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:45.497055 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:56:45.497107 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 14:56:45.497204 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":77,"threads_started":1}
I20260926 14:56:45.497851 16847 heartbeater.cc:347] Connected to a master server at 127.7.37.254:44823
I20260926 14:56:45.497946 16847 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:45.498117 16847 heartbeater.cc:511] Master 127.7.37.254:44823 requested a full tablet report, sending...
I20260926 14:56:45.498428 16528 ts_manager.cc:194] Registered new tserver with Master: 14e860c54546408da92e01c0af7d3916 (127.7.37.194:36107)
I20260926 14:56:45.498781 16528 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.37.194:56813
W20260926 14:56:45.570945 16850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:45.571115 16850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:45.571141 16850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:45.572249 16850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:45.572285 16850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:45.572316 16850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:56:45.573553 16850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:40619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:44823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:45.573758 16850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:45.573990 16850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:45.578351 16855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:45.578347 16856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:45.578377 16858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:45.578408 16850 server_base.cc:1034] running on GCE node
I20260926 14:56:45.578711 16850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:45.578914 16850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:45.580040 16850 hybrid_clock.cc:648] HybridClock initialized: now 1790434605580027 us; error 28 us; skew 500 ppm
I20260926 14:56:45.581154 16850 webserver.cc:460] Webserver started at http://127.7.37.195:43643/ using document root <none> and password file <none>
I20260926 14:56:45.581336 16850 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:45.581377 16850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:45.581492 16850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:45.582410 16850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "fe21f73724714e07af4fed27b5dbf724"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "08abbb87fab0424bcba546653e2ba5c2"
server_key_iv: "98b17c0a7b2050caa877f7da0c974eaa"
server_key_version: "encryptionkey@0"
I20260926 14:56:45.582718 16850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "fe21f73724714e07af4fed27b5dbf724"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "08abbb87fab0424bcba546653e2ba5c2"
server_key_iv: "98b17c0a7b2050caa877f7da0c974eaa"
server_key_version: "encryptionkey@0"
I20260926 14:56:45.583846 16850 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:45.584478 16864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:45.584610 16850 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:45.584683 16850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "fe21f73724714e07af4fed27b5dbf724"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "08abbb87fab0424bcba546653e2ba5c2"
server_key_iv: "98b17c0a7b2050caa877f7da0c974eaa"
server_key_version: "encryptionkey@0"
I20260926 14:56:45.584745 16850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:45.602396 16850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:45.602605 16850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:45.602710 16850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:45.602875 16850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:45.603145 16850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:45.603178 16850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:45.603201 16850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:45.603219 16850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:45.608278 16850 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:37319
I20260926 14:56:45.608311 16977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:37319 every 8 connection(s)
I20260926 14:56:45.608558 16850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:56:45.608889 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 16850
I20260926 14:56:45.608984 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:56:45.609179 7319 external_mini_cluster.cc:1428] Setting key 228191add09a6861e18f6c4f14018fe8
I20260926 14:56:45.613240 16567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:45.609093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:34833 (local address 127.7.37.254:44823)
0926 14:56:45.609149 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:45.609151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:45.609367 (+ 216us) server_negotiation.cc:407] Connection header received
0926 14:56:45.609877 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:45.609877 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:45.609889 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:45.609950 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:45.610496 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:45.610973 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:45.611974 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:45.612209 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:45.612270 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:45.613014 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:45.613016 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:45.613016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:45.613020 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:45.613036 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:45.613039 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:45.613091 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:45.613161 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 14:56:45.613183 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 14:56:45.613401 16980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:45.609086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:44823 (local address 127.7.37.195:34833)
0926 14:56:45.609263 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:45.609296 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:45.609784 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:45.610002 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:45.610005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:45.610012 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:45.610434 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:45.610438 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:45.611034 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:45.611035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:45.611910 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:45.611912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:45.612284 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:45.612284 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:45.612425 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:45.612869 (+ 444us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:56:45.612870 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:45.612894 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:56:45.612914 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:45.613101 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:45.613104 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:45.613106 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:56:45.613152 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 14:56:45.613228 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260926 14:56:45.613787 16978 heartbeater.cc:347] Connected to a master server at 127.7.37.254:44823
I20260926 14:56:45.613857 16978 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:45.614002 16978 heartbeater.cc:511] Master 127.7.37.254:44823 requested a full tablet report, sending...
I20260926 14:56:45.614328 16528 ts_manager.cc:194] Registered new tserver with Master: fe21f73724714e07af4fed27b5dbf724 (127.7.37.195:37319)
I20260926 14:56:45.614655 16528 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.37.195:34833
I20260926 14:56:45.620639 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 14:56:45.622258 16567 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:45.621869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45358 (local address 127.7.37.254:44823)
0926 14:56:45.621911 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:45.621913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:45.622002 (+ 89us) server_negotiation.cc:407] Connection header received
0926 14:56:45.622057 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:45.622057 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:45.622066 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:45.622107 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:45.622206 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45358: BlockingRecv error: recv got EOF from 127.0.0.1:45358 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 14:56:45.622378 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 16588
I20260926 14:56:45.626523 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 16719
I20260926 14:56:45.629918 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 16850
I20260926 14:56:45.633203 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 16497
2026-09-26T14:56:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (554 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-26T14:56:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:45Z Disabled control of system clock
I20260926 14:56:45.656652 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:37757
--webserver_interface=127.7.37.254
--webserver_port=0
--builtin_ntp_servers=127.7.37.212:46601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:37757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 14:56:45.731637 16989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:45.731843 16989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:45.731866 16989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:45.732931 16989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:45.732965 16989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:45.732977 16989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:45.732988 16989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:45.732998 16989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:56:45.734232 16989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:46601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:37757
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:37757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:45.734390 16989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:45.734588 16989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:45.738798 16994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:45.738818 16995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:45.738883 16989 server_base.cc:1034] running on GCE node
W20260926 14:56:45.738888 16997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:45.739174 16989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:45.739426 16989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:45.740576 16989 hybrid_clock.cc:648] HybridClock initialized: now 1790434605740560 us; error 27 us; skew 500 ppm
I20260926 14:56:45.741592 16989 webserver.cc:460] Webserver started at http://127.7.37.254:36171/ using document root <none> and password file <none>
I20260926 14:56:45.741768 16989 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:45.741806 16989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:45.741884 16989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:45.742769 16989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "f6c74127cff84a2e8be7168847428ab4"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "64d4fe7f991f2236bd09c4ae2add6859"
server_key_iv: "f8595c39982692753efeb8db5dbd5a9c"
server_key_version: "encryptionkey@0"
I20260926 14:56:45.743023 16989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "f6c74127cff84a2e8be7168847428ab4"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "64d4fe7f991f2236bd09c4ae2add6859"
server_key_iv: "f8595c39982692753efeb8db5dbd5a9c"
server_key_version: "encryptionkey@0"
I20260926 14:56:45.744113 16989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 14:56:45.744678 17003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:45.744812 16989 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:56:45.744871 16989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "f6c74127cff84a2e8be7168847428ab4"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "64d4fe7f991f2236bd09c4ae2add6859"
server_key_iv: "f8595c39982692753efeb8db5dbd5a9c"
server_key_version: "encryptionkey@0"
I20260926 14:56:45.744932 16989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:45.747874 16989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:45.748090 16989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:45.748193 16989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:45.751634 16989 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:37757
I20260926 14:56:45.751662 17055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:37757 every 8 connection(s)
I20260926 14:56:45.751935 16989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:56:45.752449 17056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:45.754729 17056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4: Bootstrap starting.
I20260926 14:56:45.755216 17056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:45.755456 17056 log.cc:824] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:45.756052 17056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4: No bootstrap required, opened a new log
I20260926 14:56:45.757477 17056 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6c74127cff84a2e8be7168847428ab4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37757 } }
I20260926 14:56:45.757596 17056 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:45.757615 17056 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6c74127cff84a2e8be7168847428ab4, State: Initialized, Role: FOLLOWER
I20260926 14:56:45.757750 17056 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6c74127cff84a2e8be7168847428ab4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37757 } }
I20260926 14:56:45.757807 17056 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:56:45.757827 17056 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:56:45.757869 17056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:45.758324 17056 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6c74127cff84a2e8be7168847428ab4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37757 } }
I20260926 14:56:45.758401 17056 leader_election.cc:302] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6c74127cff84a2e8be7168847428ab4; no voters:
I20260926 14:56:45.758529 17056 leader_election.cc:288] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:56:45.758569 17059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:45.758709 17059 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4 [term 1 LEADER]: Becoming Leader. State: Replica: f6c74127cff84a2e8be7168847428ab4, State: Running, Role: LEADER
I20260926 14:56:45.758752 17056 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:45.758790 17059 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6c74127cff84a2e8be7168847428ab4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37757 } }
I20260926 14:56:45.759147 17061 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6c74127cff84a2e8be7168847428ab4. Latest consensus state: current_term: 1 leader_uuid: "f6c74127cff84a2e8be7168847428ab4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6c74127cff84a2e8be7168847428ab4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37757 } } }
I20260926 14:56:45.759222 17061 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:45.759327 17060 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6c74127cff84a2e8be7168847428ab4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6c74127cff84a2e8be7168847428ab4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37757 } } }
I20260926 14:56:45.759399 17067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:56:45.759429 17060 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:45.759702 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 16989
I20260926 14:56:45.759815 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:56:45.759903 17067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:56:45.759987 7319 external_mini_cluster.cc:1428] Setting key 4efed455b335081c9723ee8400f74273
I20260926 14:56:45.761699 17067 catalog_manager.cc:1347] Generated new cluster ID: 6499a696724042f5b4ad3a7e5d60862c
I20260926 14:56:45.761750 17067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:56:45.762043 17078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:45.760496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55124 (local address 127.7.37.254:37757)
0926 14:56:45.760675 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:45.760682 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:45.760701 (+ 19us) server_negotiation.cc:407] Connection header received
0926 14:56:45.760778 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:45.760784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:45.760824 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:45.760949 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:45.761522 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:45.761526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:45.761526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:45.761530 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:45.761564 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:45.761576 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:45.761817 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:45.761845 (+ 28us) server_negotiation.cc:299] Negotiation successful
0926 14:56:45.761896 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260926 14:56:45.786474 17067 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:56:45.786610 17067 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:45.798130 17067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6c74127cff84a2e8be7168847428ab4: Generated new TSK 0
I20260926 14:56:45.798273 17067 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:56:45.807170 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:37757
--builtin_ntp_servers=127.7.37.212:46601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 14:56:45.882663 17080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:45.882851 17080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:45.882879 17080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:45.884020 17080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:45.884055 17080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:45.884089 17080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:56:45.885373 17080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:46601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:37757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:45.885583 17080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:45.885824 17080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:45.890152 17086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:45.890157 17088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:45.890151 17085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:45.890177 17080 server_base.cc:1034] running on GCE node
I20260926 14:56:45.890499 17080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:45.890674 17080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:45.891788 17080 hybrid_clock.cc:648] HybridClock initialized: now 1790434605891774 us; error 23 us; skew 500 ppm
I20260926 14:56:45.892910 17080 webserver.cc:460] Webserver started at http://127.7.37.193:35793/ using document root <none> and password file <none>
I20260926 14:56:45.893108 17080 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:45.893154 17080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:45.893275 17080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:45.894191 17080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "5c23f2528c28420da83dc5c112634e5c"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "ccb0262adc48611cb925ced64f8632cc"
server_key_iv: "f136590d702495201824e8c6d6bc119a"
server_key_version: "encryptionkey@0"
I20260926 14:56:45.894495 17080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "5c23f2528c28420da83dc5c112634e5c"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "ccb0262adc48611cb925ced64f8632cc"
server_key_iv: "f136590d702495201824e8c6d6bc119a"
server_key_version: "encryptionkey@0"
I20260926 14:56:45.895594 17080 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:56:45.896301 17094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:45.896449 17080 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:45.896524 17080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "5c23f2528c28420da83dc5c112634e5c"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "ccb0262adc48611cb925ced64f8632cc"
server_key_iv: "f136590d702495201824e8c6d6bc119a"
server_key_version: "encryptionkey@0"
I20260926 14:56:45.896587 17080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:45.899580 17080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:45.899837 17080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:45.899950 17080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:45.900122 17080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:45.900370 17080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:45.900400 17080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:45.900424 17080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:45.900447 17080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:45.905406 17080 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:45205
I20260926 14:56:45.905468 17207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:45205 every 8 connection(s)
I20260926 14:56:45.905707 17080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:56:45.911348 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 17080
I20260926 14:56:45.911432 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:56:45.911582 7319 external_mini_cluster.cc:1428] Setting key e69a0c00f6624b36930fe4fc65ac18e6
I20260926 14:56:45.912390 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:37757
--builtin_ntp_servers=127.7.37.212:46601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 14:56:45.913262 17078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:45.906224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:37505 (local address 127.7.37.254:37757)
0926 14:56:45.906263 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:45.906265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:45.906431 (+ 166us) server_negotiation.cc:407] Connection header received
0926 14:56:45.906802 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:45.906803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:45.906809 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:45.906853 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:45.913024 (+ 6171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:45.913025 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:45.913026 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:45.913031 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:45.913048 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:45.913053 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:45.913116 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:45.913182 (+ 66us) server_negotiation.cc:299] Negotiation successful
0926 14:56:45.913196 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 14:56:45.913383 17210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:45.906210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:37757 (local address 127.7.37.193:37505)
0926 14:56:45.906345 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:45.906372 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:45.906733 (+ 361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:45.906899 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:45.906902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:45.906909 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:45.912888 (+ 5979us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:56:45.912888 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:45.912913 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:56:45.912939 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:45.913123 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:45.913125 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:45.913127 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:56:45.913173 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 14:56:45.913221 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":62,"threads_started":1}
I20260926 14:56:45.913825 17208 heartbeater.cc:347] Connected to a master server at 127.7.37.254:37757
I20260926 14:56:45.913925 17208 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:45.914057 17208 heartbeater.cc:511] Master 127.7.37.254:37757 requested a full tablet report, sending...
I20260926 14:56:45.914489 17020 ts_manager.cc:194] Registered new tserver with Master: 5c23f2528c28420da83dc5c112634e5c (127.7.37.193:45205)
W20260926 14:56:45.987501 17211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:45.987670 17211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:45.987699 17211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:45.988837 17211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:45.988875 17211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:45.988909 17211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:56:45.990163 17211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:46601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:37757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:45.990358 17211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:45.990587 17211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:45.994867 17217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:45.994902 17219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:45.994979 17216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:45.994992 17211 server_base.cc:1034] running on GCE node
I20260926 14:56:45.995226 17211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:45.995433 17211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:45.996562 17211 hybrid_clock.cc:648] HybridClock initialized: now 1790434605996552 us; error 25 us; skew 500 ppm
I20260926 14:56:45.997593 17211 webserver.cc:460] Webserver started at http://127.7.37.194:33295/ using document root <none> and password file <none>
I20260926 14:56:45.997776 17211 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:45.997820 17211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:45.997936 17211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:45.998947 17211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "a793f4400b77466e9aa5b62119061654"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "cbdd21c424e335d9f03edfcb160272c3"
server_key_iv: "df78f9a88cf32b73849160a687fbb1f9"
server_key_version: "encryptionkey@0"
I20260926 14:56:45.999259 17211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "a793f4400b77466e9aa5b62119061654"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "cbdd21c424e335d9f03edfcb160272c3"
server_key_iv: "df78f9a88cf32b73849160a687fbb1f9"
server_key_version: "encryptionkey@0"
I20260926 14:56:46.000468 17211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 14:56:46.001156 17225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:46.001325 17211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:46.001397 17211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "a793f4400b77466e9aa5b62119061654"
format_stamp: "Formatted at 2026-09-26 14:56:45 on dist-test-slave-f9xb"
server_key: "cbdd21c424e335d9f03edfcb160272c3"
server_key_iv: "df78f9a88cf32b73849160a687fbb1f9"
server_key_version: "encryptionkey@0"
I20260926 14:56:46.001443 17211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:46.004384 17211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:46.004565 17211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:46.004655 17211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:46.004794 17211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:46.004992 17211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:46.005018 17211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:46.005035 17211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:46.005051 17211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:46.009927 17211 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:34913
I20260926 14:56:46.009985 17338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:34913 every 8 connection(s)
I20260926 14:56:46.010201 17211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:56:46.012702 17078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:46.010734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:52809 (local address 127.7.37.254:37757)
0926 14:56:46.010789 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:46.010791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:46.010969 (+ 178us) server_negotiation.cc:407] Connection header received
0926 14:56:46.011397 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:46.011398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:46.011404 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:46.011451 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:46.012474 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:46.012475 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:46.012476 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:46.012480 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:46.012495 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:46.012499 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:46.012547 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:46.012615 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 14:56:46.012635 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 14:56:46.012842 17341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:46.010716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:37757 (local address 127.7.37.194:52809)
0926 14:56:46.010874 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:46.010901 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:46.011308 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:46.011509 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:46.011513 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:46.011520 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:46.012346 (+ 826us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:56:46.012346 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:46.012370 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:56:46.012394 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:46.012552 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:46.012555 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:46.012558 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:46.012619 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 14:56:46.012671 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260926 14:56:46.013242 17339 heartbeater.cc:347] Connected to a master server at 127.7.37.254:37757
I20260926 14:56:46.013316 17339 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:46.013438 17339 heartbeater.cc:511] Master 127.7.37.254:37757 requested a full tablet report, sending...
I20260926 14:56:46.013705 17020 ts_manager.cc:194] Registered new tserver with Master: a793f4400b77466e9aa5b62119061654 (127.7.37.194:34913)
I20260926 14:56:46.016320 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 17211
I20260926 14:56:46.016408 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:56:46.016556 7319 external_mini_cluster.cc:1428] Setting key e1f70bee0ec91ff3da14f5e13c2858e9
I20260926 14:56:46.017251 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:37757
--builtin_ntp_servers=127.7.37.212:46601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 14:56:46.091583 17342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:46.091782 17342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:46.091811 17342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:46.092935 17342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:46.092970 17342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:46.092994 17342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:56:46.094269 17342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:46601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:37757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:46.094460 17342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:46.094703 17342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:46.098985 17347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:46.098990 17350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:46.098999 17348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:46.099261 17342 server_base.cc:1034] running on GCE node
I20260926 14:56:46.099426 17342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:46.099619 17342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:46.100755 17342 hybrid_clock.cc:648] HybridClock initialized: now 1790434606100729 us; error 39 us; skew 500 ppm
I20260926 14:56:46.101796 17342 webserver.cc:460] Webserver started at http://127.7.37.195:39593/ using document root <none> and password file <none>
I20260926 14:56:46.101990 17342 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:46.102035 17342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:46.102151 17342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:46.103073 17342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "0790594da40b46ffa64e9c9a5bccfb81"
format_stamp: "Formatted at 2026-09-26 14:56:46 on dist-test-slave-f9xb"
server_key: "edc53fe87ad6ab0cc18c31432900126d"
server_key_iv: "573d55166752e965e0a0a23e2ce0e14c"
server_key_version: "encryptionkey@0"
I20260926 14:56:46.103377 17342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "0790594da40b46ffa64e9c9a5bccfb81"
format_stamp: "Formatted at 2026-09-26 14:56:46 on dist-test-slave-f9xb"
server_key: "edc53fe87ad6ab0cc18c31432900126d"
server_key_iv: "573d55166752e965e0a0a23e2ce0e14c"
server_key_version: "encryptionkey@0"
I20260926 14:56:46.104616 17342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 14:56:46.105238 17356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:46.105374 17342 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:46.105445 17342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "0790594da40b46ffa64e9c9a5bccfb81"
format_stamp: "Formatted at 2026-09-26 14:56:46 on dist-test-slave-f9xb"
server_key: "edc53fe87ad6ab0cc18c31432900126d"
server_key_iv: "573d55166752e965e0a0a23e2ce0e14c"
server_key_version: "encryptionkey@0"
I20260926 14:56:46.105506 17342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:46.108533 17342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:46.108745 17342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:46.108852 17342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:46.109014 17342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:46.109253 17342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:46.109282 17342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:46.109306 17342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:46.109329 17342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:46.114391 17342 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:39289
I20260926 14:56:46.114427 17469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:39289 every 8 connection(s)
I20260926 14:56:46.114684 17342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:56:46.117144 17078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:46.115255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:40129 (local address 127.7.37.254:37757)
0926 14:56:46.115300 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:46.115302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:46.115495 (+ 193us) server_negotiation.cc:407] Connection header received
0926 14:56:46.115915 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:46.115916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:46.115921 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:46.115965 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:46.116967 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:46.116968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:46.116968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:56:46.116971 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:46.116979 (+ 8us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:56:46.116982 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:46.117014 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:46.117059 (+ 45us) server_negotiation.cc:299] Negotiation successful
0926 14:56:46.117070 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 14:56:46.117249 17472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:46.115239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:37757 (local address 127.7.37.195:40129)
0926 14:56:46.115400 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:46.115427 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:46.115838 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:46.116017 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:46.116020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:46.116028 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:46.116859 (+ 831us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:56:46.116860 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:46.116884 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:56:46.116908 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:46.117017 (+ 109us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:46.117019 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:46.117021 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:56:46.117064 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 14:56:46.117102 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
I20260926 14:56:46.117619 17470 heartbeater.cc:347] Connected to a master server at 127.7.37.254:37757
I20260926 14:56:46.117686 17470 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:46.117779 17470 heartbeater.cc:511] Master 127.7.37.254:37757 requested a full tablet report, sending...
I20260926 14:56:46.118018 17020 ts_manager.cc:194] Registered new tserver with Master: 0790594da40b46ffa64e9c9a5bccfb81 (127.7.37.195:39289)
I20260926 14:56:46.121032 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 17342
I20260926 14:56:46.121140 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:56:46.121273 7319 external_mini_cluster.cc:1428] Setting key c7ef15c250fc8126eba61b69032a3847
I20260926 14:56:46.121912 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 14:56:46.123448 17078 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:56:46.123024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55132 (local address 127.7.37.254:37757)
0926 14:56:46.123056 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:46.123058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:46.123180 (+ 122us) server_negotiation.cc:407] Connection header received
0926 14:56:46.123237 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:46.123238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:46.123242 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:46.123282 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:46.123397 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55132: BlockingRecv error: recv got EOF from 127.0.0.1:55132 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
I20260926 14:56:46.123520 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 17080
I20260926 14:56:46.127976 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 17211
I20260926 14:56:46.131078 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 17342
I20260926 14:56:46.134173 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 16989
2026-09-26T14:56:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (500 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:56:46 dist-test-slave-f9xb krb5kdc[17480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:56:46 dist-test-slave-f9xb krb5kdc[17480](info): set up 2 sockets
Sep 26 14:56:46 dist-test-slave-f9xb krb5kdc[17480](info): commencing operation
krb5kdc: starting...
W20260926 14:56:47.080034 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:56:47 dist-test-slave-f9xb krb5kdc[17480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:56:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:56:47.180446 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:43255
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:37457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:43255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:47.255051 17493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:47.255227 17493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:47.255247 17493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:47.256330 17493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:56:47.256361 17493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:47.256372 17493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:47.256383 17493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:56:47.256394 17493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:56:47.257736 17493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:37457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:43255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:43255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:47.257900 17493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:47.258044 17493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:47.262315 17498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:47.262331 17501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:47.262535 17493 server_base.cc:1034] running on GCE node
W20260926 14:56:47.262331 17499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:47.262804 17493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:47.263054 17493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:47.264181 17493 hybrid_clock.cc:648] HybridClock initialized: now 1790434607264170 us; error 29 us; skew 500 ppm
Sep 26 14:56:47 dist-test-slave-f9xb krb5kdc[17480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434607, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:47.266347 17493 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:56:47.266739 17493 webserver.cc:460] Webserver started at http://127.7.37.254:41407/ using document root <none> and password file <none>
I20260926 14:56:47.266911 17493 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:47.266955 17493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:47.267066 17493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:47.268004 17493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "e99f65fa1aa844bbb06163c93b5a8915"
format_stamp: "Formatted at 2026-09-26 14:56:47 on dist-test-slave-f9xb"
server_key: "0d2d8ac66e00c5725ec84f798b7b82bf"
server_key_iv: "c164f53923bab255355351e0cfaafce3"
server_key_version: "encryptionkey@0"
I20260926 14:56:47.268291 17493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "e99f65fa1aa844bbb06163c93b5a8915"
format_stamp: "Formatted at 2026-09-26 14:56:47 on dist-test-slave-f9xb"
server_key: "0d2d8ac66e00c5725ec84f798b7b82bf"
server_key_iv: "c164f53923bab255355351e0cfaafce3"
server_key_version: "encryptionkey@0"
I20260926 14:56:47.269387 17493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:56:47.270002 17508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:47.270152 17493 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:56:47.270223 17493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "e99f65fa1aa844bbb06163c93b5a8915"
format_stamp: "Formatted at 2026-09-26 14:56:47 on dist-test-slave-f9xb"
server_key: "0d2d8ac66e00c5725ec84f798b7b82bf"
server_key_iv: "c164f53923bab255355351e0cfaafce3"
server_key_version: "encryptionkey@0"
I20260926 14:56:47.270278 17493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:47.283998 17493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:47.285784 17493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:47.285899 17493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:47.289307 17493 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:43255
I20260926 14:56:47.289336 17560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:43255 every 8 connection(s)
I20260926 14:56:47.289598 17493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:56:47.290385 17561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:47.292295 17561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915: Bootstrap starting.
I20260926 14:56:47.292784 17561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:47.293017 17561 log.cc:824] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:47.293568 17561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915: No bootstrap required, opened a new log
I20260926 14:56:47.293774 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 17493
I20260926 14:56:47.293849 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:56:47.294004 7319 external_mini_cluster.cc:1428] Setting key 2707a0ec442aef5874e26553a151a895
I20260926 14:56:47.295359 17561 raft_consensus.cc:348] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e99f65fa1aa844bbb06163c93b5a8915" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43255 } }
I20260926 14:56:47.295509 17561 raft_consensus.cc:374] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:47.295576 17561 raft_consensus.cc:729] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e99f65fa1aa844bbb06163c93b5a8915, State: Initialized, Role: FOLLOWER
I20260926 14:56:47.295710 17561 consensus_queue.cc:260] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e99f65fa1aa844bbb06163c93b5a8915" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43255 } }
I20260926 14:56:47.295809 17561 raft_consensus.cc:388] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:56:47.295845 17561 raft_consensus.cc:482] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:56:47.295887 17561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:47.296770 17561 raft_consensus.cc:504] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e99f65fa1aa844bbb06163c93b5a8915" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43255 } }
I20260926 14:56:47.296866 17561 leader_election.cc:302] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e99f65fa1aa844bbb06163c93b5a8915; no voters:
I20260926 14:56:47.296995 17561 leader_election.cc:288] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:56:47.297020 17566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:47.297192 17566 raft_consensus.cc:686] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915 [term 1 LEADER]: Becoming Leader. State: Replica: e99f65fa1aa844bbb06163c93b5a8915, State: Running, Role: LEADER
I20260926 14:56:47.297199 17561 sys_catalog.cc:564] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:56:47.297353 17566 consensus_queue.cc:237] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e99f65fa1aa844bbb06163c93b5a8915" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43255 } }
I20260926 14:56:47.298727 17568 sys_catalog.cc:455] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e99f65fa1aa844bbb06163c93b5a8915. Latest consensus state: current_term: 1 leader_uuid: "e99f65fa1aa844bbb06163c93b5a8915" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e99f65fa1aa844bbb06163c93b5a8915" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43255 } } }
I20260926 14:56:47.298734 17567 sys_catalog.cc:455] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e99f65fa1aa844bbb06163c93b5a8915" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e99f65fa1aa844bbb06163c93b5a8915" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43255 } } }
I20260926 14:56:47.299629 17567 sys_catalog.cc:458] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915 [sys.catalog]: This master's current role is: LEADER
I20260926 14:56:47.299613 17568 sys_catalog.cc:458] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915 [sys.catalog]: This master's current role is: LEADER
W20260926 14:56:47.298820 17581 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 14:56:47.299793 17581 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 26 14:56:47 dist-test-slave-f9xb krb5kdc[17480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:47.304258 17565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.294403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51914 (local address 127.7.37.254:43255)
0926 14:56:47.294596 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:47.294604 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:47.294635 (+ 31us) server_negotiation.cc:407] Connection header received
0926 14:56:47.295335 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:47.295342 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:47.295398 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:47.295535 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:47.296211 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.297515 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.298491 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.299004 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.299098 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:47.301430 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:47.301433 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:47.301433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:47.301442 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:47.302822 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:47.303181 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:47.303182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:47.303183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:47.303270 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:47.303436 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:47.303437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:47.303437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:47.303628 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:47.303737 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:47.303870 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 14:56:47.303964 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":86,"threads_started":1}
I20260926 14:56:47.310408 17583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:56:47.310566 17583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:56:47.311842 17583 catalog_manager.cc:1347] Generated new cluster ID: 0c8419fdbb5b407bb3e3dfe324f482f0
I20260926 14:56:47.311887 17583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:56:47.324940 17583 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:56:47.325628 17583 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:56:47.342505 17583 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915: Generated new TSK 0
I20260926 14:56:47.342645 17583 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
I20260926 14:56:47.370333 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:43255
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:37457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:47.445034 17587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:47.445204 17587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:47.445232 17587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:47.446298 17587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:47.446334 17587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:47.446368 17587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:56:47.447775 17587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:37457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:43255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:47.447984 17587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:47.448160 17587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:47.452370 17593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:47.452348 17595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:47.452355 17592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:47.452564 17587 server_base.cc:1034] running on GCE node
I20260926 14:56:47.452741 17587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:47.452932 17587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:47.454058 17587 hybrid_clock.cc:648] HybridClock initialized: now 1790434607454046 us; error 28 us; skew 500 ppm
Sep 26 14:56:47 dist-test-slave-f9xb krb5kdc[17480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434607, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:47.456283 17587 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:56:47.456609 17587 webserver.cc:460] Webserver started at http://127.7.37.193:38711/ using document root <none> and password file <none>
I20260926 14:56:47.456772 17587 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:47.456809 17587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:47.456890 17587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:47.457732 17587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "97261c417c6c4d72b7ad23dc022583e1"
format_stamp: "Formatted at 2026-09-26 14:56:47 on dist-test-slave-f9xb"
server_key: "7c01327e2b63b28199dfb2dbaafab2fc"
server_key_iv: "6847ec35079f604535fc9be8a3976bd7"
server_key_version: "encryptionkey@0"
I20260926 14:56:47.457993 17587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "97261c417c6c4d72b7ad23dc022583e1"
format_stamp: "Formatted at 2026-09-26 14:56:47 on dist-test-slave-f9xb"
server_key: "7c01327e2b63b28199dfb2dbaafab2fc"
server_key_iv: "6847ec35079f604535fc9be8a3976bd7"
server_key_version: "encryptionkey@0"
I20260926 14:56:47.459036 17587 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 14:56:47.459609 17602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:47.459757 17587 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:47.459830 17587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "97261c417c6c4d72b7ad23dc022583e1"
format_stamp: "Formatted at 2026-09-26 14:56:47 on dist-test-slave-f9xb"
server_key: "7c01327e2b63b28199dfb2dbaafab2fc"
server_key_iv: "6847ec35079f604535fc9be8a3976bd7"
server_key_version: "encryptionkey@0"
I20260926 14:56:47.459879 17587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:47.480832 17587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:47.482297 17587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:47.482389 17587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:47.482535 17587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:47.482755 17587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:47.482781 17587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:47.482798 17587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:47.482815 17587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:47.487897 17587 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:34151
I20260926 14:56:47.487954 17715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:34151 every 8 connection(s)
I20260926 14:56:47.488183 17587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
Sep 26 14:56:47 dist-test-slave-f9xb krb5kdc[17480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434607, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:47.494109 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 17587
I20260926 14:56:47.494212 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:56:47.494400 7319 external_mini_cluster.cc:1428] Setting key 562b1854014998abb3f598f180d098d6
I20260926 14:56:47.495929 17565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.488763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:50877 (local address 127.7.37.254:43255)
0926 14:56:47.488815 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:47.488816 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:47.488963 (+ 147us) server_negotiation.cc:407] Connection header received
0926 14:56:47.489381 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:47.489382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:47.489394 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:47.489431 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:47.490207 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.490681 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.491529 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.491725 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.491806 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:47.493459 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:47.493460 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:47.493460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:47.493467 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:47.494788 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:47.495153 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:47.495154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:47.495155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:47.495208 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:47.495389 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:47.495389 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:47.495390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:47.495504 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:47.495558 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:47.495773 (+ 215us) server_negotiation.cc:299] Negotiation successful
0926 14:56:47.495845 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 14:56:47.496017 17718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.488730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43255 (local address 127.7.37.193:50877)
0926 14:56:47.488859 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:47.488894 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:47.489297 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:47.489478 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:47.489480 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:47.489975 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:47.490142 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:47.490146 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.490745 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.490745 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:47.491473 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:47.491474 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.491805 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.491806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:47.491890 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:47.492225 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:47.492232 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:47.493384 (+ 1152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:47.494851 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:47.494856 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:47.494858 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:47.495091 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:47.495263 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:47.495263 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:47.495263 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:47.495343 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:47.495550 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:47.495552 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:47.495654 (+ 102us) client_negotiation.cc:769] Sending connection context
0926 14:56:47.495722 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 14:56:47.495867 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":70,"threads_started":1}
I20260926 14:56:47.496426 17716 heartbeater.cc:347] Connected to a master server at 127.7.37.254:43255
I20260926 14:56:47.496496 17716 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:47.496639 17716 heartbeater.cc:511] Master 127.7.37.254:43255 requested a full tablet report, sending...
I20260926 14:56:47.497094 17525 ts_manager.cc:194] Registered new tserver with Master: 97261c417c6c4d72b7ad23dc022583e1 (127.7.37.193:34151)
I20260926 14:56:47.497906 17525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:50877
WARNING: no policy specified for kudu/127.7.37.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
I20260926 14:56:47.515875 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:43255
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:37457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:47.589270 17721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:47.589435 17721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:47.589463 17721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:47.590535 17721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:47.590572 17721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:47.590605 17721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:56:47.591987 17721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:37457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:43255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:47.592190 17721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:47.592363 17721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:47.596621 17729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:47.596647 17726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:47.596733 17727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:47.596733 17721 server_base.cc:1034] running on GCE node
I20260926 14:56:47.596983 17721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:47.597173 17721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:47.598299 17721 hybrid_clock.cc:648] HybridClock initialized: now 1790434607598286 us; error 28 us; skew 500 ppm
Sep 26 14:56:47 dist-test-slave-f9xb krb5kdc[17480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434607, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:47.600497 17721 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:56:47.600867 17721 webserver.cc:460] Webserver started at http://127.7.37.194:43675/ using document root <none> and password file <none>
I20260926 14:56:47.601038 17721 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:47.601076 17721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:47.601197 17721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:47.602074 17721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "70144408be6943b592f03f24cee8f305"
format_stamp: "Formatted at 2026-09-26 14:56:47 on dist-test-slave-f9xb"
server_key: "fceae6f35ed4e8acaf321f1f59c62a64"
server_key_iv: "725a951a9ceb61b058380f92c59b6504"
server_key_version: "encryptionkey@0"
I20260926 14:56:47.602376 17721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "70144408be6943b592f03f24cee8f305"
format_stamp: "Formatted at 2026-09-26 14:56:47 on dist-test-slave-f9xb"
server_key: "fceae6f35ed4e8acaf321f1f59c62a64"
server_key_iv: "725a951a9ceb61b058380f92c59b6504"
server_key_version: "encryptionkey@0"
I20260926 14:56:47.603443 17721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 14:56:47.604279 17736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:47.604440 17721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 14:56:47.604513 17721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "70144408be6943b592f03f24cee8f305"
format_stamp: "Formatted at 2026-09-26 14:56:47 on dist-test-slave-f9xb"
server_key: "fceae6f35ed4e8acaf321f1f59c62a64"
server_key_iv: "725a951a9ceb61b058380f92c59b6504"
server_key_version: "encryptionkey@0"
I20260926 14:56:47.604576 17721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:47.633608 17721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:47.635424 17721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:47.635532 17721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:47.635713 17721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:47.635972 17721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:47.636003 17721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:47.636029 17721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:47.636045 17721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:47.640872 17721 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:38609
I20260926 14:56:47.640923 17849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:38609 every 8 connection(s)
I20260926 14:56:47.641155 17721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
Sep 26 14:56:47 dist-test-slave-f9xb krb5kdc[17480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434607, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:47.648581 17565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.641724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:49067 (local address 127.7.37.254:43255)
0926 14:56:47.641779 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:47.641780 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:47.641928 (+ 148us) server_negotiation.cc:407] Connection header received
0926 14:56:47.642342 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:47.642343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:47.642352 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:47.642389 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:47.643180 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.643648 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.644597 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.644778 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.644820 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:47.646479 (+ 1659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:47.646480 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:47.646481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:47.646486 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:47.647611 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:47.647954 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:47.647955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:47.647955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:47.648000 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:47.648169 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:47.648169 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:47.648169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:47.648254 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:47.648295 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:47.648468 (+ 173us) server_negotiation.cc:299] Negotiation successful
0926 14:56:47.648523 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 14:56:47.648746 17852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.641680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43255 (local address 127.7.37.194:49067)
0926 14:56:47.641824 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:47.641860 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:47.642264 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:47.642437 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:47.642439 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:47.642950 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:47.643114 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:47.643118 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.643730 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.643732 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:47.644524 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:47.644526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.644827 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.644828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:47.644913 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:47.645254 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:47.645261 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:47.646399 (+ 1138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:47.647678 (+ 1279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:47.647681 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:47.647683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:47.647897 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:47.648041 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:47.648041 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:47.648042 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:47.648132 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:47.648291 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:47.648293 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:47.648391 (+ 98us) client_negotiation.cc:769] Sending connection context
0926 14:56:47.648457 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 14:56:47.648556 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
I20260926 14:56:47.649122 17850 heartbeater.cc:347] Connected to a master server at 127.7.37.254:43255
I20260926 14:56:47.649194 17850 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:47.649358 17850 heartbeater.cc:511] Master 127.7.37.254:43255 requested a full tablet report, sending...
I20260926 14:56:47.649523 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 17721
I20260926 14:56:47.649614 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:56:47.649694 17525 ts_manager.cc:194] Registered new tserver with Master: 70144408be6943b592f03f24cee8f305 (127.7.37.194:38609)
I20260926 14:56:47.649796 7319 external_mini_cluster.cc:1428] Setting key d6c0ccd974fec2868518353573ec004e
I20260926 14:56:47.650100 17525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:49067
WARNING: no policy specified for kudu/127.7.37.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
I20260926 14:56:47.670634 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:43255
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:37457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:56:47.744390 17855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:56:47.744550 17855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:56:47.744570 17855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:56:47.745658 17855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:56:47.745695 17855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:56:47.745720 17855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:56:47.747051 17855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:37457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:43255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:56:47.747248 17855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:56:47.747432 17855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:56:47.751772 17861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:56:47.751776 17863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:47.751914 17855 server_base.cc:1034] running on GCE node
W20260926 14:56:47.751786 17860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:56:47.752190 17855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:56:47.752396 17855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:56:47.753545 17855 hybrid_clock.cc:648] HybridClock initialized: now 1790434607753529 us; error 30 us; skew 500 ppm
Sep 26 14:56:47 dist-test-slave-f9xb krb5kdc[17480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434607, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:56:47.755637 17855 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:56:47.756042 17855 webserver.cc:460] Webserver started at http://127.7.37.195:42649/ using document root <none> and password file <none>
I20260926 14:56:47.756235 17855 fs_manager.cc:359] Metadata directory not provided
I20260926 14:56:47.756284 17855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:56:47.756398 17855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:56:47.757287 17855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "c4b732b5d0e742d7a25f4ad369d6c9f5"
format_stamp: "Formatted at 2026-09-26 14:56:47 on dist-test-slave-f9xb"
server_key: "8ed60cede8817606c418592a14a4cd20"
server_key_iv: "de8def864851af0ff4b31100479f46f3"
server_key_version: "encryptionkey@0"
I20260926 14:56:47.757572 17855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "c4b732b5d0e742d7a25f4ad369d6c9f5"
format_stamp: "Formatted at 2026-09-26 14:56:47 on dist-test-slave-f9xb"
server_key: "8ed60cede8817606c418592a14a4cd20"
server_key_iv: "de8def864851af0ff4b31100479f46f3"
server_key_version: "encryptionkey@0"
I20260926 14:56:47.758661 17855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 14:56:47.759256 17870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:47.759393 17855 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:56:47.759466 17855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "c4b732b5d0e742d7a25f4ad369d6c9f5"
format_stamp: "Formatted at 2026-09-26 14:56:47 on dist-test-slave-f9xb"
server_key: "8ed60cede8817606c418592a14a4cd20"
server_key_iv: "de8def864851af0ff4b31100479f46f3"
server_key_version: "encryptionkey@0"
I20260926 14:56:47.759526 17855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:56:47.769237 17855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:56:47.770701 17855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:56:47.770823 17855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:56:47.770980 17855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:56:47.771220 17855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:56:47.771250 17855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:47.771273 17855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:56:47.771297 17855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:56:47.776428 17855 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:37015
I20260926 14:56:47.776463 17983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:37015 every 8 connection(s)
I20260926 14:56:47.776702 17855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
Sep 26 14:56:47 dist-test-slave-f9xb krb5kdc[17480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434607, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:56:47.783905 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 17855
I20260926 14:56:47.783989 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:56:47.784143 7319 external_mini_cluster.cc:1428] Setting key a4fc26c7c2ab5c2cee3273003e8ee70a
I20260926 14:56:47.784217 17565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.777241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:50025 (local address 127.7.37.254:43255)
0926 14:56:47.777288 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:47.777290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:47.777449 (+ 159us) server_negotiation.cc:407] Connection header received
0926 14:56:47.777862 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:47.777864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:47.777875 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:47.777924 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:47.778734 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.779209 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.780153 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.780339 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.780391 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:47.782078 (+ 1687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:47.782079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:47.782080 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:47.782087 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:47.783082 (+ 995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:47.783395 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:47.783396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:47.783397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:47.783445 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:47.783639 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:47.783640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:47.783641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:47.783758 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:47.783815 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:47.784075 (+ 260us) server_negotiation.cc:299] Negotiation successful
0926 14:56:47.784139 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 14:56:47.784636 17986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.777234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:43255 (local address 127.7.37.195:50025)
0926 14:56:47.777354 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:47.777384 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:47.777782 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:47.777980 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:47.777984 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:47.778492 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:56:47.778650 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:47.778654 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.779279 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.779280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:47.780081 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:47.780083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.780399 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.780400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:47.780499 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:47.780850 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:56:47.780856 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:56:47.781999 (+ 1143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:56:47.783146 (+ 1147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:47.783152 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:47.783153 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:47.783336 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:47.783496 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:56:47.783496 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:56:47.783497 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:56:47.783587 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:56:47.783807 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:56:47.783812 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:56:47.783927 (+ 115us) client_negotiation.cc:769] Sending connection context
0926 14:56:47.784303 (+ 376us) client_negotiation.cc:240] Negotiation successful
0926 14:56:47.784449 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":59,"threads_started":1}
I20260926 14:56:47.785010 17984 heartbeater.cc:347] Connected to a master server at 127.7.37.254:43255
I20260926 14:56:47.785079 17984 heartbeater.cc:464] Registering TS with master...
I20260926 14:56:47.785225 17984 heartbeater.cc:511] Master 127.7.37.254:43255 requested a full tablet report, sending...
I20260926 14:56:47.785513 17524 ts_manager.cc:194] Registered new tserver with Master: c4b732b5d0e742d7a25f4ad369d6c9f5 (127.7.37.195:37015)
I20260926 14:56:47.785853 17524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:50025
I20260926 14:56:47.795526 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 14:56:47.801857 17565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.796747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51916 (local address 127.7.37.254:43255)
0926 14:56:47.796780 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:47.796782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:47.796880 (+ 98us) server_negotiation.cc:407] Connection header received
0926 14:56:47.796924 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:47.796924 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:47.796933 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:47.796975 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:56:47.797529 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.798021 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.798770 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.798941 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.798989 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:47.799984 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:56:47.799986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:56:47.799986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:56:47.799992 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:56:47.800924 (+ 932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:47.801202 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:47.801203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:47.801203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:47.801248 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:56:47.801395 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:56:47.801396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:56:47.801397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:56:47.801513 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:56:47.801593 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:47.801714 (+ 121us) server_negotiation.cc:299] Negotiation successful
0926 14:56:47.801779 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260926 14:56:47.802979 17524 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51916:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:56:47.804437 17524 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:56:47.809919 17993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.806123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:38609 (local address 127.0.0.1:59964)
0926 14:56:47.806354 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:47.806363 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:47.806408 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:47.806717 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:47.806718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:47.806720 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:47.806769 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:47.806771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.808131 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.808131 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:47.808850 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:47.808852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.809697 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.809698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:47.809749 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:47.809758 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 14:56:47.809812 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 14:56:47.809855 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":40,"threads_started":1}
I20260926 14:56:47.809938 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.806374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59964 (local address 127.7.37.194:38609)
0926 14:56:47.806511 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:47.806515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:47.806526 (+ 11us) server_negotiation.cc:407] Connection header received
0926 14:56:47.806562 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:47.806564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:47.806579 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:47.806649 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:47.806843 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.808046 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.808936 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.809631 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.809701 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:47.809752 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:47.809830 (+ 78us) server_negotiation.cc:299] Negotiation successful
0926 14:56:47.809870 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260926 14:56:47.810178 17994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.806217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58724 (local address 127.7.37.195:37015)
0926 14:56:47.806396 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:47.806400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:47.806410 (+ 10us) server_negotiation.cc:407] Connection header received
0926 14:56:47.806439 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:47.806441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:47.806454 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:47.806520 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:47.806703 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.807662 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.809136 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.809755 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.809813 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:47.809860 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:47.810080 (+ 220us) server_negotiation.cc:299] Negotiation successful
0926 14:56:47.810116 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":78,"threads_started":1}
I20260926 14:56:47.810025 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.806111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:37015 (local address 127.0.0.1:58724)
0926 14:56:47.806204 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:47.806239 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:47.806299 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:47.806532 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:47.806534 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:47.806536 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:47.806598 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:47.806599 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.807814 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.807815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:47.809068 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:47.809070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.809820 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.809821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:47.809882 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:47.809885 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:47.809929 (+ 44us) client_negotiation.cc:240] Negotiation successful
0926 14:56:47.809964 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":37,"thread_start_us":50,"threads_started":1}
I20260926 14:56:47.810853 17995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.806279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:34151 (local address 127.0.0.1:48250)
0926 14:56:47.806589 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:47.806598 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:47.806634 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:47.806886 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:47.806886 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:47.806888 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:47.806915 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:47.806916 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.807763 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.807764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:47.809068 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:47.809070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.810629 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.810630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:47.810691 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:47.810694 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:47.810752 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 14:56:47.810789 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":47,"thread_start_us":34,"threads_started":1}
I20260926 14:56:47.810877 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.806536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48250 (local address 127.7.37.193:34151)
0926 14:56:47.806755 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:47.806759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:47.806768 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:56:47.806800 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:47.806802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:47.806816 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:47.806880 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:47.806960 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.807666 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.809894 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.810489 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.810547 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:47.810599 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:47.810773 (+ 174us) server_negotiation.cc:299] Negotiation successful
0926 14:56:47.810810 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":64,"threads_started":1}
I20260926 14:56:47.810889 17918 tablet_service.cc:1467] Processing CreateTablet for tablet 2e7057316ae141968b9ab7c5380cb9a8 (DEFAULT_TABLE table=test-table [id=4cde144da395432694fb1da066c67523]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:47.810891 17784 tablet_service.cc:1467] Processing CreateTablet for tablet 2e7057316ae141968b9ab7c5380cb9a8 (DEFAULT_TABLE table=test-table [id=4cde144da395432694fb1da066c67523]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:47.811184 17784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e7057316ae141968b9ab7c5380cb9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:47.811184 17918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e7057316ae141968b9ab7c5380cb9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:47.811527 17650 tablet_service.cc:1467] Processing CreateTablet for tablet 2e7057316ae141968b9ab7c5380cb9a8 (DEFAULT_TABLE table=test-table [id=4cde144da395432694fb1da066c67523]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:56:47.811769 17650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e7057316ae141968b9ab7c5380cb9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:56:47.814289 17998 tablet_bootstrap.cc:492] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1: Bootstrap starting.
I20260926 14:56:47.814790 17998 tablet_bootstrap.cc:654] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:47.815210 17998 log.cc:824] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:47.815480 18000 tablet_bootstrap.cc:492] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305: Bootstrap starting.
I20260926 14:56:47.815817 17998 tablet_bootstrap.cc:492] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1: No bootstrap required, opened a new log
I20260926 14:56:47.815877 17998 ts_tablet_manager.cc:1397] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:56:47.815877 17999 tablet_bootstrap.cc:492] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5: Bootstrap starting.
I20260926 14:56:47.816052 18000 tablet_bootstrap.cc:654] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:47.816516 17999 tablet_bootstrap.cc:654] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5: Neither blocks nor log segments found. Creating new log.
I20260926 14:56:47.816545 18000 log.cc:824] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:47.816987 17999 log.cc:824] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5: Log is configured to *not* fsync() on all Append() calls
I20260926 14:56:47.817240 18000 tablet_bootstrap.cc:492] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305: No bootstrap required, opened a new log
I20260926 14:56:47.817350 18000 ts_tablet_manager.cc:1397] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:56:47.817648 17999 tablet_bootstrap.cc:492] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5: No bootstrap required, opened a new log
I20260926 14:56:47.817549 17998 raft_consensus.cc:348] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b732b5d0e742d7a25f4ad369d6c9f5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 37015 } } peers { permanent_uuid: "70144408be6943b592f03f24cee8f305" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38609 } } peers { permanent_uuid: "97261c417c6c4d72b7ad23dc022583e1" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34151 } }
I20260926 14:56:47.817698 17998 raft_consensus.cc:374] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:47.817713 17999 ts_tablet_manager.cc:1397] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:56:47.817728 17998 raft_consensus.cc:729] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97261c417c6c4d72b7ad23dc022583e1, State: Initialized, Role: FOLLOWER
I20260926 14:56:47.817830 17998 consensus_queue.cc:260] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b732b5d0e742d7a25f4ad369d6c9f5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 37015 } } peers { permanent_uuid: "70144408be6943b592f03f24cee8f305" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38609 } } peers { permanent_uuid: "97261c417c6c4d72b7ad23dc022583e1" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34151 } }
I20260926 14:56:47.818104 17998 ts_tablet_manager.cc:1428] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:56:47.818224 17716 heartbeater.cc:503] Master 127.7.37.254:43255 was elected leader, sending a full tablet report...
I20260926 14:56:47.819084 18000 raft_consensus.cc:348] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b732b5d0e742d7a25f4ad369d6c9f5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 37015 } } peers { permanent_uuid: "70144408be6943b592f03f24cee8f305" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38609 } } peers { permanent_uuid: "97261c417c6c4d72b7ad23dc022583e1" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34151 } }
I20260926 14:56:47.819311 18000 raft_consensus.cc:374] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:47.819365 18000 raft_consensus.cc:729] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70144408be6943b592f03f24cee8f305, State: Initialized, Role: FOLLOWER
I20260926 14:56:47.819468 18000 consensus_queue.cc:260] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b732b5d0e742d7a25f4ad369d6c9f5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 37015 } } peers { permanent_uuid: "70144408be6943b592f03f24cee8f305" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38609 } } peers { permanent_uuid: "97261c417c6c4d72b7ad23dc022583e1" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34151 } }
I20260926 14:56:47.819469 17999 raft_consensus.cc:348] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b732b5d0e742d7a25f4ad369d6c9f5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 37015 } } peers { permanent_uuid: "70144408be6943b592f03f24cee8f305" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38609 } } peers { permanent_uuid: "97261c417c6c4d72b7ad23dc022583e1" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34151 } }
I20260926 14:56:47.819599 17999 raft_consensus.cc:374] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:56:47.819638 17999 raft_consensus.cc:729] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b732b5d0e742d7a25f4ad369d6c9f5, State: Initialized, Role: FOLLOWER
I20260926 14:56:47.819705 18000 ts_tablet_manager.cc:1428] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:56:47.819723 17850 heartbeater.cc:503] Master 127.7.37.254:43255 was elected leader, sending a full tablet report...
I20260926 14:56:47.819800 17999 consensus_queue.cc:260] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b732b5d0e742d7a25f4ad369d6c9f5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 37015 } } peers { permanent_uuid: "70144408be6943b592f03f24cee8f305" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38609 } } peers { permanent_uuid: "97261c417c6c4d72b7ad23dc022583e1" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34151 } }
I20260926 14:56:47.820041 17984 heartbeater.cc:503] Master 127.7.37.254:43255 was elected leader, sending a full tablet report...
I20260926 14:56:47.820237 17999 ts_tablet_manager.cc:1428] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 14:56:47.888388 18004 raft_consensus.cc:482] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:56:47.888470 18004 raft_consensus.cc:504] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b732b5d0e742d7a25f4ad369d6c9f5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 37015 } } peers { permanent_uuid: "70144408be6943b592f03f24cee8f305" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38609 } } peers { permanent_uuid: "97261c417c6c4d72b7ad23dc022583e1" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34151 } }
I20260926 14:56:47.888772 18004 leader_election.cc:288] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b732b5d0e742d7a25f4ad369d6c9f5 (127.7.37.195:37015), 70144408be6943b592f03f24cee8f305 (127.7.37.194:38609)
I20260926 14:56:47.891196 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.888994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:52431 (local address 127.7.37.194:38609)
0926 14:56:47.889030 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:47.889032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:47.889039 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:56:47.889060 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:47.889061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:47.889070 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:47.889116 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:47.889234 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.889683 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.890548 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.890913 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.890972 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:47.890979 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:47.891109 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 14:56:47.891139 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:56:47.891198 17718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.888910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:38609 (local address 127.7.37.193:52431)
0926 14:56:47.888949 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:47.888960 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:47.889000 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:47.889127 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:47.889128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:47.889129 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:47.889182 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:47.889183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.889750 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.889751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:47.890464 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:47.890466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.890984 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.890985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:47.891052 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:47.891055 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:47.891097 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 14:56:47.891134 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":18}
I20260926 14:56:47.891270 17994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.889002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:47607 (local address 127.7.37.195:37015)
0926 14:56:47.889040 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:47.889042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:47.889123 (+ 81us) server_negotiation.cc:407] Connection header received
0926 14:56:47.889177 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:47.889178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:47.889185 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:47.889226 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:56:47.889341 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.889773 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.890626 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.890996 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.891053 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:47.891060 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:47.891179 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 14:56:47.891212 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 14:56:47.891296 18007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.888912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:37015 (local address 127.7.37.193:47607)
0926 14:56:47.889062 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:47.889070 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:56:47.889123 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:56:47.889233 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:56:47.889234 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:56:47.889235 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:56:47.889293 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:47.889294 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.889836 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.889837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:47.890556 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:56:47.890558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.891061 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.891062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:56:47.891125 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:47.891128 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:56:47.891170 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 14:56:47.891205 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":9,"thread_start_us":73,"threads_started":1}
I20260926 14:56:47.891515 17804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e7057316ae141968b9ab7c5380cb9a8" candidate_uuid: "97261c417c6c4d72b7ad23dc022583e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70144408be6943b592f03f24cee8f305" is_pre_election: true
W20260926 14:56:47.891584 17851 tablet.cc:2369] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:56:47.891639 17804 raft_consensus.cc:2454] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97261c417c6c4d72b7ad23dc022583e1 in term 0.
I20260926 14:56:47.891736 17938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e7057316ae141968b9ab7c5380cb9a8" candidate_uuid: "97261c417c6c4d72b7ad23dc022583e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b732b5d0e742d7a25f4ad369d6c9f5" is_pre_election: true
I20260926 14:56:47.891824 17604 leader_election.cc:302] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70144408be6943b592f03f24cee8f305, 97261c417c6c4d72b7ad23dc022583e1; no voters:
I20260926 14:56:47.891885 17938 raft_consensus.cc:2454] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97261c417c6c4d72b7ad23dc022583e1 in term 0.
I20260926 14:56:47.891912 18004 raft_consensus.cc:2781] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:56:47.891966 18004 raft_consensus.cc:482] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:56:47.891988 18004 raft_consensus.cc:3037] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:47.892604 18004 raft_consensus.cc:504] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b732b5d0e742d7a25f4ad369d6c9f5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 37015 } } peers { permanent_uuid: "70144408be6943b592f03f24cee8f305" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38609 } } peers { permanent_uuid: "97261c417c6c4d72b7ad23dc022583e1" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34151 } }
I20260926 14:56:47.892704 18004 leader_election.cc:288] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [CANDIDATE]: Term 1 election: Requested vote from peers c4b732b5d0e742d7a25f4ad369d6c9f5 (127.7.37.195:37015), 70144408be6943b592f03f24cee8f305 (127.7.37.194:38609)
I20260926 14:56:47.892836 17938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e7057316ae141968b9ab7c5380cb9a8" candidate_uuid: "97261c417c6c4d72b7ad23dc022583e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b732b5d0e742d7a25f4ad369d6c9f5"
I20260926 14:56:47.892863 17804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e7057316ae141968b9ab7c5380cb9a8" candidate_uuid: "97261c417c6c4d72b7ad23dc022583e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70144408be6943b592f03f24cee8f305"
I20260926 14:56:47.892899 17938 raft_consensus.cc:3037] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:47.892928 17804 raft_consensus.cc:3037] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:56:47.893513 17938 raft_consensus.cc:2454] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97261c417c6c4d72b7ad23dc022583e1 in term 1.
I20260926 14:56:47.893559 17804 raft_consensus.cc:2454] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97261c417c6c4d72b7ad23dc022583e1 in term 1.
I20260926 14:56:47.893684 17603 leader_election.cc:302] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97261c417c6c4d72b7ad23dc022583e1, c4b732b5d0e742d7a25f4ad369d6c9f5; no voters:
I20260926 14:56:47.893774 18004 raft_consensus.cc:2781] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:56:47.893898 18004 raft_consensus.cc:686] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [term 1 LEADER]: Becoming Leader. State: Replica: 97261c417c6c4d72b7ad23dc022583e1, State: Running, Role: LEADER
I20260926 14:56:47.893972 18004 consensus_queue.cc:237] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b732b5d0e742d7a25f4ad369d6c9f5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 37015 } } peers { permanent_uuid: "70144408be6943b592f03f24cee8f305" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38609 } } peers { permanent_uuid: "97261c417c6c4d72b7ad23dc022583e1" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34151 } }
I20260926 14:56:47.894614 17525 catalog_manager.cc:5495] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97261c417c6c4d72b7ad23dc022583e1 (127.7.37.193). New cstate: current_term: 1 leader_uuid: "97261c417c6c4d72b7ad23dc022583e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b732b5d0e742d7a25f4ad369d6c9f5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 37015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70144408be6943b592f03f24cee8f305" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97261c417c6c4d72b7ad23dc022583e1" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34151 } health_report { overall_health: HEALTHY } } }
I20260926 14:56:47.899327 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.897433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48258 (local address 127.7.37.193:34151)
0926 14:56:47.897465 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 14:56:47.897467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:56:47.897475 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:56:47.897534 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:56:47.897535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:56:47.897548 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:56:47.897590 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:56:47.897673 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.898122 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.898809 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:56:47.898978 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:56:47.899024 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:56:47.899125 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:56:47.899203 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:56:47.899251 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 14:56:47.899262 (+ 11us) server_negotiation.cc:299] Negotiation successful
0926 14:56:47.899274 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 14:56:47.900640 17804 raft_consensus.cc:1261] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305 [term 1 FOLLOWER]: Refusing update from remote peer 97261c417c6c4d72b7ad23dc022583e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:47.900657 17938 raft_consensus.cc:1261] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5 [term 1 FOLLOWER]: Refusing update from remote peer 97261c417c6c4d72b7ad23dc022583e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:56:47.900908 18008 consensus_queue.cc:1035] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70144408be6943b592f03f24cee8f305" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:47.901011 18004 consensus_queue.cc:1035] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b732b5d0e742d7a25f4ad369d6c9f5" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 37015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:56:47.902935 18011 mvcc.cc:205] Tried to move back new op lower bound from 7333620153959084032 to 7333620153934176256. Current Snapshot: MvccSnapshot[applied={T|T < 7333620153959084032}]
I20260926 14:56:47.903002 18012 mvcc.cc:205] Tried to move back new op lower bound from 7333620153959084032 to 7333620153934176256. Current Snapshot: MvccSnapshot[applied={T|T < 7333620153959084032}]
I20260926 14:56:47.904769 17525 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:47.904831 17525 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:56:47.905397 17525 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e99f65fa1aa844bbb06163c93b5a8915: Sending DeleteTablet for 3 replicas of tablet 2e7057316ae141968b9ab7c5380cb9a8
I20260926 14:56:47.905676 17784 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e7057316ae141968b9ab7c5380cb9a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:47 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:59964
I20260926 14:56:47.905702 17918 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e7057316ae141968b9ab7c5380cb9a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:47 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:58724
I20260926 14:56:47.905885 18021 tablet_replica.cc:330] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305: stopping tablet replica
I20260926 14:56:47.905939 17650 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e7057316ae141968b9ab7c5380cb9a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:56:47 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:48250
I20260926 14:56:47.905968 18021 raft_consensus.cc:2229] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:47.906054 18021 raft_consensus.cc:2258] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:47.906106 18023 tablet_replica.cc:330] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1: stopping tablet replica
I20260926 14:56:47.906203 18023 raft_consensus.cc:2229] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:56:47.906217 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 17587
I20260926 14:56:47.906311 18023 raft_consensus.cc:2258] T 2e7057316ae141968b9ab7c5380cb9a8 P 97261c417c6c4d72b7ad23dc022583e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:47.906430 18021 ts_tablet_manager.cc:1905] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:47.907698 18022 tablet_replica.cc:330] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5: stopping tablet replica
I20260926 14:56:47.907789 18022 raft_consensus.cc:2229] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:56:47.907871 18022 raft_consensus.cc:2258] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:56:47.907881 18021 ts_tablet_manager.cc:1918] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:56:47.907932 18021 log.cc:1196] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/2e7057316ae141968b9ab7c5380cb9a8
I20260926 14:56:47.908223 18022 ts_tablet_manager.cc:1905] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:56:47.908215 18021 ts_tablet_manager.cc:1939] T 2e7057316ae141968b9ab7c5380cb9a8 P 70144408be6943b592f03f24cee8f305: Deleting consensus metadata
I20260926 14:56:47.909021 17510 catalog_manager.cc:4841] TS 70144408be6943b592f03f24cee8f305 (127.7.37.194:38609): tablet 2e7057316ae141968b9ab7c5380cb9a8 (table test-table [id=4cde144da395432694fb1da066c67523]) successfully deleted
I20260926 14:56:47.909308 18022 ts_tablet_manager.cc:1918] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:56:47.909363 18022 log.cc:1196] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/2e7057316ae141968b9ab7c5380cb9a8
I20260926 14:56:47.909566 18022 ts_tablet_manager.cc:1939] T 2e7057316ae141968b9ab7c5380cb9a8 P c4b732b5d0e742d7a25f4ad369d6c9f5: Deleting consensus metadata
I20260926 14:56:47.909876 17509 catalog_manager.cc:4841] TS c4b732b5d0e742d7a25f4ad369d6c9f5 (127.7.37.195:37015): tablet 2e7057316ae141968b9ab7c5380cb9a8 (table test-table [id=4cde144da395432694fb1da066c67523]) successfully deleted
W20260926 14:56:47.911585 17509 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.37.193:34151 (error 108)
I20260926 14:56:47.911818 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 17721
I20260926 14:56:47.912299 17995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:56:47.911863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:34151 (local address 127.0.0.1:48274)
0926 14:56:47.912238 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0926 14:56:47.912276 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.193:34151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356}
W20260926 14:56:47.912390 17509 catalog_manager.cc:4584] TS 97261c417c6c4d72b7ad23dc022583e1 (127.7.37.193:34151): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e7057316ae141968b9ab7c5380cb9a8: Network error: Client connection negotiation failed: client connection to 127.7.37.193:34151: connect: Connection refused (error 111)
I20260926 14:56:47.916250 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 17855
I20260926 14:56:47.919795 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 17493
2026-09-26T14:56:47Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1786 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:56:47 dist-test-slave-f9xb krb5kdc[18027](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:56:47 dist-test-slave-f9xb krb5kdc[18027](info): set up 2 sockets
Sep 26 14:56:47 dist-test-slave-f9xb krb5kdc[18027](info): commencing operation
krb5kdc: starting...
W20260926 14:56:48.864135 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:56:48 dist-test-slave-f9xb krb5kdc[18027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:56:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:56:48Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.37.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.37.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.37.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/rangeradmin_127.7.37.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.37.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/rangeradmin_127.7.37.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.37.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.37.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.37.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/rangerlookup_127.7.37.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.37.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/rangerlookup_127.7.37.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.37.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.37.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.37.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/HTTP_127.7.37.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.37.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/HTTP_127.7.37.212@KRBTEST.COM.keytab.
I20260926 14:56:48.999403 7319 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskrMzbz4/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/postgres -l logfile start
2026-09-26 14:56:51.173 UTC [18057] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 14:56:51.173 UTC [18057] LOG: listening on IPv4 address "127.7.37.212", port 40091
2026-09-26 14:56:51.176 UTC [18057] LOG: listening on Unix socket "/tmp/.s.PGSQL.40091"
2026-09-26 14:56:51.195 UTC [18060] LOG: database system was shut down at 2026-09-26 14:56:49 UTC
2026-09-26 14:56:51.198 UTC [18057] LOG: database system is ready to accept connections
I20260926 14:56:52.068008 7319 mini_postgres.cc:96] Postgres bound to 40091
2026-09-26 14:56:52.073 UTC [18068] FATAL: database "slave" does not exist
127.7.37.212:40091 - accepting connections
I20260926 14:56:52.073827 7319 mini_ranger.cc:161] Starting Ranger...
I20260926 14:56:52.086813 7319 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 14:56:52.467618 7319 mini_ranger.cc:87] Created ranger Postgres database
I20260926 14:56:52.467685 7319 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ranger-admin
2026-09-26 14:56:52,736 [I] DB FLAVOR :POSTGRES
2026-09-26 14:56:52,737 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 14:56:52,737 [I] Checking connection..
2026-09-26 14:56:52,737 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 14:56:52,967 [I] Checking connection passed.
2026-09-26 14:56:52,967 [I] --------- Verifying version history table ---------
2026-09-26 14:56:52,967 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 14:56:53,190 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 14:56:53,191 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 14:56:53,418 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 14:56:53,419 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 14:56:53,419 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 14:56:53.636 UTC [18132] WARNING: there is no transaction in progress
2026-09-26 14:56:53,641 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 14:56:53,642 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 14:56:53,879 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 14:56:53,879 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 14:56:53,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 14:56:54,096 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 14:56:54,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-f9xb.c.gcp-upstream.internal','N') ;"
2026-09-26 14:56:54,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 14:56:54,765 [I] Table x_portal_user does not exist in database ranger
2026-09-26 14:56:54,766 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 14:56:54,975 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 14:56:54,976 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 14:56:55,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 14:56:55,415 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 14:56:55,415 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 14:56:56.277 UTC [18240] WARNING: there is no transaction in progress
2026-09-26 14:56:56.292 UTC [18240] WARNING: there is no transaction in progress
2026-09-26 14:56:56.307 UTC [18240] WARNING: there is no transaction in progress
2026-09-26 14:56:56.321 UTC [18240] WARNING: there is no transaction in progress
2026-09-26 14:56:56.335 UTC [18240] WARNING: there is no transaction in progress
2026-09-26 14:56:56.350 UTC [18240] WARNING: there is no transaction in progress
2026-09-26 14:56:56.398 UTC [18240] WARNING: there is no transaction in progress
2026-09-26 14:56:56.403 UTC [18240] WARNING: there is no transaction in progress
2026-09-26 14:56:56.416 UTC [18240] WARNING: there is no transaction in progress
2026-09-26 14:56:56.424 UTC [18240] WARNING: there is no transaction in progress
2026-09-26 14:56:56.433 UTC [18240] WARNING: there is no transaction in progress
2026-09-26 14:56:56.445 UTC [18240] WARNING: there is no transaction in progress
2026-09-26 14:56:56.453 UTC [18240] WARNING: there is no transaction in progress
2026-09-26 14:56:56.459 UTC [18240] WARNING: there is no transaction in progress
2026-09-26 14:56:56.464 UTC [18240] WARNING: there is no transaction in progress
2026-09-26 14:56:56.469 UTC [18240] WARNING: there is no transaction in progress
2026-09-26 14:56:57,014 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 14:56:57,014 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 14:56:57,232 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 14:56:57,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 14:56:57,460 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 14:56:57,460 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 14:56:57,706 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 14:56:57,707 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 14:56:57,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 14:56:58,159 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-f9xb.c.gcp-upstream.internal';"
2026-09-26 14:56:58,383 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 14:56:58,383 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 14:56:58,611 [I] DB_PATCHES have already been applied
I20260926 14:56:58.616734 7319 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ranger-admin:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskrMzbz4/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 14:56:58.616791 7319 mini_ranger.cc:193] Using host: 127.7.37.212
I20260926 14:56:58.619393 7319 mini_ranger.cc:227] Ranger admin URL: http://127.7.37.212:52211
Sep 26, 2026 2:56:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 2:56:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 2:56:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 2:56:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 2:56:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 2:56:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 2:56:59 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 14:56:59.603193 7319 mini_ranger.cc:160] Time spent starting Ranger: real 7.529s user 0.000s sys 0.007s
Sep 26, 2026 2:57:17 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 2:57:17 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 26, 2026 2:57:17 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 2:57:17 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 2:57:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 2:57:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 2:57:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 2:57:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 2:57:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 2:57:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 2:57:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 2:57:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 2:57:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 2:57:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 2:57:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 2:57:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 2:57:17 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 2:57:17 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 14:57:18.454455 7319 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.37.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.37.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.37.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/rangerkms_127.7.37.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.37.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/rangerkms_127.7.37.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.37.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.37.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.37.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/HTTP_127.7.37.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.37.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/HTTP_127.7.37.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 14:57:18 dist-test-slave-f9xb krb5kdc[18027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434638, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 14:57:18.529000 7319 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 14:57:18.536396 7319 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 14:57:19.282873 7319 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 14:57:19.282953 7319 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ranger-kms
I20260926 14:57:19.282979 7319 mini_ranger_kms.cc:227] Using postgres at 127.7.37.212:40091
2026-09-26 14:57:19,436 [I] DB FLAVOR :POSTGRES
2026-09-26 14:57:19,436 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 14:57:19,436 [I] Checking connection
2026-09-26 14:57:19,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 14:57:19,659 [I] connection success
2026-09-26 14:57:19,660 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 14:57:19,660 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 14:57:19,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 14:57:19,869 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 14:57:19,869 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 14:57:19,870 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 14:57:19,870 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:40091/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 14:57:20,105 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 14:57:20.493088 7319 mini_ranger_kms.cc:310] Created kms service
I20260926 14:57:20.663702 7319 mini_ranger_kms.cc:326] Created kudu user
I20260926 14:57:20.700385 7319 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 14:57:20 dist-test-slave-f9xb krb5kdc[18027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434638, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
I20260926 14:57:21.066871 7319 mini_ranger_kms.cc:384] Added ranger policy
I20260926 14:57:21.066974 7319 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ranger-kms:/tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 14:57:21.066994 7319 mini_ranger_kms.cc:240] Using host: 127.7.37.212
I20260926 14:57:21.070045 7319 mini_ranger_kms.cc:276] Ranger KMS PID: 18420
I20260926 14:57:21.070158 7319 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.37.212:46253
Sep 26, 2026 2:57:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 2:57:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 2:57:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 2:57:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 2:57:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 2:57:22 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 14:57:23.107605 7319 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.579s user 0.008s sys 0.129s
I20260926 14:57:23.107717 7319 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 14:57:23.107813 7319 mini_ranger_kms.cc:401] 127.7.37.212:46253/kms/v1/keys
Sep 26 14:57:23 dist-test-slave-f9xb krb5kdc[18027](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790434643, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.37.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 14:57:23 dist-test-slave-f9xb krb5kdc[18027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434643, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.37.212@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
Sep 26 14:57:24 dist-test-slave-f9xb krb5kdc[18027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434643, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.37.212@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
Sep 26 14:57:26 dist-test-slave-f9xb krb5kdc[18027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:57:26.107510 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:42905
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:39373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:42905
--ranger_config_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.37.212:46253/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:57:26.196350 18471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:57:26.196589 18471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:57:26.196615 18471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:57:26.198231 18471 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:57:26.198282 18471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:57:26.198299 18471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:57:26.198315 18471 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:57:26.198330 18471 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:57:26.200479 18471 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:39373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.37.212:46253/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:42905
--ranger_config_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:42905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:57:26.200704 18471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:57:26.200933 18471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:57:26.205436 18476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:57:26.205526 18479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:57:26.205693 18471 server_base.cc:1034] running on GCE node
W20260926 14:57:26.205423 18477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:57:26.205922 18471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:57:26.206192 18471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:57:26.207341 18471 hybrid_clock.cc:648] HybridClock initialized: now 1790434646207328 us; error 28 us; skew 500 ppm
Sep 26 14:57:26 dist-test-slave-f9xb krb5kdc[18027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434646, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:57:26.209651 18471 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:57:26.210017 18471 webserver.cc:460] Webserver started at http://127.7.37.254:34601/ using document root <none> and password file <none>
I20260926 14:57:26.210193 18471 fs_manager.cc:359] Metadata directory not provided
I20260926 14:57:26.210230 18471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:57:26.210351 18471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 14:57:26 dist-test-slave-f9xb krb5kdc[18027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434646, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
I20260926 14:57:26.253955 18471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "52cae4e38bf140488e8c3b3ddcb9bdd5"
format_stamp: "Formatted at 2026-09-26 14:57:26 on dist-test-slave-f9xb"
server_key: "7266639ca4ae7fb7c638ba109f5b335f"
server_key_iv: "f2e5bc43b3a410c3ccb86fdb41d6a600"
server_key_version: "kuduclusterkey@0"
I20260926 14:57:26.254303 18471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "52cae4e38bf140488e8c3b3ddcb9bdd5"
format_stamp: "Formatted at 2026-09-26 14:57:26 on dist-test-slave-f9xb"
server_key: "7266639ca4ae7fb7c638ba109f5b335f"
server_key_iv: "f2e5bc43b3a410c3ccb86fdb41d6a600"
server_key_version: "kuduclusterkey@0"
I20260926 14:57:26.255579 18471 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:57:26.266665 18486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:57:26.266848 18471 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 14:57:26.266935 18471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "52cae4e38bf140488e8c3b3ddcb9bdd5"
format_stamp: "Formatted at 2026-09-26 14:57:26 on dist-test-slave-f9xb"
server_key: "7266639ca4ae7fb7c638ba109f5b335f"
server_key_iv: "f2e5bc43b3a410c3ccb86fdb41d6a600"
server_key_version: "kuduclusterkey@0"
I20260926 14:57:26.267009 18471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:57:26.282511 18471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:57:26.284241 18471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:57:26.284358 18471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:57:26.288375 18471 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:42905
I20260926 14:57:26.288408 18538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:42905 every 8 connection(s)
I20260926 14:57:26.288692 18471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:57:26.289268 18539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:57:26.291293 18539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5: Bootstrap starting.
I20260926 14:57:26.291791 18539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5: Neither blocks nor log segments found. Creating new log.
I20260926 14:57:26.292001 18539 log.cc:824] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5: Log is configured to *not* fsync() on all Append() calls
I20260926 14:57:26.292037 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 18471
I20260926 14:57:26.292137 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:57:26.292320 7319 external_mini_cluster.cc:1428] Setting key 584c49b68e84559dec12903ab5711975
I20260926 14:57:26.292677 18539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5: No bootstrap required, opened a new log
I20260926 14:57:26.294831 18539 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52cae4e38bf140488e8c3b3ddcb9bdd5" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42905 } }
I20260926 14:57:26.294986 18539 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:57:26.295022 18539 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52cae4e38bf140488e8c3b3ddcb9bdd5, State: Initialized, Role: FOLLOWER
I20260926 14:57:26.295173 18539 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52cae4e38bf140488e8c3b3ddcb9bdd5" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42905 } }
I20260926 14:57:26.295236 18539 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:57:26.295256 18539 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:57:26.295287 18539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:57:26.295833 18539 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52cae4e38bf140488e8c3b3ddcb9bdd5" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42905 } }
I20260926 14:57:26.295930 18539 leader_election.cc:302] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52cae4e38bf140488e8c3b3ddcb9bdd5; no voters:
I20260926 14:57:26.296062 18539 leader_election.cc:288] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:57:26.296095 18544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:57:26.296247 18544 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5 [term 1 LEADER]: Becoming Leader. State: Replica: 52cae4e38bf140488e8c3b3ddcb9bdd5, State: Running, Role: LEADER
I20260926 14:57:26.296295 18539 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:57:26.296370 18544 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52cae4e38bf140488e8c3b3ddcb9bdd5" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42905 } }
I20260926 14:57:26.296712 18545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52cae4e38bf140488e8c3b3ddcb9bdd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52cae4e38bf140488e8c3b3ddcb9bdd5" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42905 } } }
I20260926 14:57:26.296782 18545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5 [sys.catalog]: This master's current role is: LEADER
I20260926 14:57:26.296993 18546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52cae4e38bf140488e8c3b3ddcb9bdd5. Latest consensus state: current_term: 1 leader_uuid: "52cae4e38bf140488e8c3b3ddcb9bdd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52cae4e38bf140488e8c3b3ddcb9bdd5" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42905 } } }
I20260926 14:57:26.297297 18546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5 [sys.catalog]: This master's current role is: LEADER
I20260926 14:57:26.297641 18539 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 14:57:26.297962 18551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:57:26.300623 18551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 14:57:26 dist-test-slave-f9xb krb5kdc[18027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:57:26.304412 18551 catalog_manager.cc:1347] Generated new cluster ID: fd2cf39eaa1849498b2dbe70d3da89c0
I20260926 14:57:26.304466 18551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:57:26.305682 18543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:26.292775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48716 (local address 127.7.37.254:42905)
0926 14:57:26.292957 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 14:57:26.292967 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 14:57:26.292997 (+ 30us) server_negotiation.cc:407] Connection header received
0926 14:57:26.294817 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:57:26.294826 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:57:26.294887 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:57:26.295029 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:57:26.295674 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:26.297028 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:26.297982 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:26.300193 (+ 2211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:26.300353 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:57:26.302714 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:57:26.302718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:57:26.302718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:57:26.302729 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:57:26.304342 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:57:26.304673 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:57:26.304676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:57:26.304676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:57:26.304780 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:57:26.304958 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:57:26.304959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:57:26.304960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:57:26.305174 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:57:26.305292 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 14:57:26.305410 (+ 118us) server_negotiation.cc:299] Negotiation successful
0926 14:57:26.305519 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260926 14:57:26.374563 18551 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:57:26.375335 18551 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:57:26.388944 18551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5: Generated new TSK 0
I20260926 14:57:26.389096 18551 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
I20260926 14:57:26.956117 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.37.212:46253/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42905
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:39373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:57:27.038269 18584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:57:27.038444 18584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:57:27.038472 18584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:57:27.039585 18584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:57:27.039623 18584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:57:27.039655 18584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:57:27.041079 18584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:39373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.37.212:46253/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:57:27.041283 18584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:57:27.041474 18584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:57:27.046448 18592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:57:27.046777 18589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:57:27.046502 18590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:57:27.046923 18584 server_base.cc:1034] running on GCE node
I20260926 14:57:27.047216 18584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:57:27.047449 18584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:57:27.063957 18584 hybrid_clock.cc:648] HybridClock initialized: now 1790434647063940 us; error 35 us; skew 500 ppm
Sep 26 14:57:27 dist-test-slave-f9xb krb5kdc[18027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434647, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:57:27.066250 18584 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:57:27.066653 18584 webserver.cc:460] Webserver started at http://127.7.37.193:38511/ using document root <none> and password file <none>
I20260926 14:57:27.066871 18584 fs_manager.cc:359] Metadata directory not provided
I20260926 14:57:27.066916 18584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:57:27.067028 18584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 14:57:27 dist-test-slave-f9xb krb5kdc[18027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434647, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
I20260926 14:57:27.078953 18584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "60024aff442d4cbd88ff173839631ca3"
format_stamp: "Formatted at 2026-09-26 14:57:27 on dist-test-slave-f9xb"
server_key: "12eaf412b55e75931e5c80d43728f22a"
server_key_iv: "d9cf541af776dfe694c3cab0da83aab6"
server_key_version: "kuduclusterkey@0"
I20260926 14:57:27.079413 18584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "60024aff442d4cbd88ff173839631ca3"
format_stamp: "Formatted at 2026-09-26 14:57:27 on dist-test-slave-f9xb"
server_key: "12eaf412b55e75931e5c80d43728f22a"
server_key_iv: "d9cf541af776dfe694c3cab0da83aab6"
server_key_version: "kuduclusterkey@0"
I20260926 14:57:27.080833 18584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:57:27.094069 18599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:57:27.094231 18584 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:57:27.094316 18584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "60024aff442d4cbd88ff173839631ca3"
format_stamp: "Formatted at 2026-09-26 14:57:27 on dist-test-slave-f9xb"
server_key: "12eaf412b55e75931e5c80d43728f22a"
server_key_iv: "d9cf541af776dfe694c3cab0da83aab6"
server_key_version: "kuduclusterkey@0"
I20260926 14:57:27.094393 18584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:57:27.128368 18584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:57:27.131420 18584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:57:27.131626 18584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:57:27.131886 18584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:57:27.132241 18584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:57:27.132282 18584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:57:27.132313 18584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:57:27.132362 18584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:57:27.140095 18584 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:40509
I20260926 14:57:27.140566 18584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:57:27.141330 18714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:40509 every 8 connection(s)
I20260926 14:57:27.148773 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 18584
I20260926 14:57:27.148872 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:57:27.149052 7319 external_mini_cluster.cc:1428] Setting key 38c0de389f745fb93476aafe1d02d800
Sep 26 14:57:27 dist-test-slave-f9xb krb5kdc[18027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434647, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:57:27.153475 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.141987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42905 (local address 127.7.37.193:34397)
0926 14:57:27.143865 (+ 1878us) negotiation.cc:107] Waiting for socket to connect
0926 14:57:27.143911 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 14:57:27.144514 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:57:27.144970 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:57:27.144975 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:57:27.145219 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:57:27.145487 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:57:27.145493 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.146478 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.146479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:57:27.147718 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:57:27.147721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.148278 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.148279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:57:27.148423 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:57:27.148959 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:57:27.148968 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:57:27.150283 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:57:27.152029 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:57:27.152035 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:57:27.152037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:57:27.152296 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:57:27.152493 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:57:27.152493 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:57:27.152494 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:57:27.152604 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:57:27.152855 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:57:27.152861 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:57:27.153013 (+ 152us) client_negotiation.cc:769] Sending connection context
0926 14:57:27.153097 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 14:57:27.153255 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1834,"thread_start_us":65,"threads_started":1}
I20260926 14:57:27.154276 18717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.141982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:34397 (local address 127.7.37.254:42905)
0926 14:57:27.144722 (+ 2740us) server_negotiation.cc:206] Beginning negotiation
0926 14:57:27.144724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:57:27.144732 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:57:27.144757 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:57:27.144758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:57:27.144771 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:57:27.144842 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:57:27.145601 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.146373 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.147840 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.148147 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.148213 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:57:27.150370 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:57:27.150372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:57:27.150373 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:57:27.150379 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:57:27.151938 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:57:27.152376 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:57:27.152377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:57:27.152377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:57:27.152436 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:57:27.152675 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:57:27.152676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:57:27.152676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:57:27.152793 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:57:27.154050 (+ 1257us) server_negotiation.cc:1035] Waiting for connection context
0926 14:57:27.154094 (+ 44us) server_negotiation.cc:299] Negotiation successful
0926 14:57:27.154183 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2710,"thread_start_us":75,"threads_started":1}
I20260926 14:57:27.154790 18715 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42905
I20260926 14:57:27.154872 18715 heartbeater.cc:464] Registering TS with master...
I20260926 14:57:27.155040 18715 heartbeater.cc:511] Master 127.7.37.254:42905 requested a full tablet report, sending...
I20260926 14:57:27.155584 18503 ts_manager.cc:194] Registered new tserver with Master: 60024aff442d4cbd88ff173839631ca3 (127.7.37.193:40509)
I20260926 14:57:27.156566 18503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:34397
WARNING: no policy specified for kudu/127.7.37.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
I20260926 14:57:27.184351 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.37.212:46253/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42905
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:39373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:57:27.279448 18721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:57:27.279631 18721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:57:27.279660 18721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:57:27.280756 18721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:57:27.280794 18721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:57:27.280825 18721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:57:27.282224 18721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:39373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.37.212:46253/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:57:27.282428 18721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:57:27.282624 18721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:57:27.287469 18731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:57:27.287693 18721 server_base.cc:1034] running on GCE node
W20260926 14:57:27.287767 18728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:57:27.287483 18729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:57:27.288022 18721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:57:27.288230 18721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:57:27.289372 18721 hybrid_clock.cc:648] HybridClock initialized: now 1790434647289358 us; error 35 us; skew 500 ppm
Sep 26 14:57:27 dist-test-slave-f9xb krb5kdc[18027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434647, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:57:27.292191 18721 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:57:27.292662 18721 webserver.cc:460] Webserver started at http://127.7.37.194:45129/ using document root <none> and password file <none>
I20260926 14:57:27.292912 18721 fs_manager.cc:359] Metadata directory not provided
I20260926 14:57:27.292977 18721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:57:27.293152 18721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 14:57:27 dist-test-slave-f9xb krb5kdc[18027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434647, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
I20260926 14:57:27.305959 18721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "0ddf2f3ad0c24c8c80d3f15e16c3d1bf"
format_stamp: "Formatted at 2026-09-26 14:57:27 on dist-test-slave-f9xb"
server_key: "97550618d308b5ce67346ef8ca1c02f2"
server_key_iv: "9f10adbff3113c0909201ca286ae5ba5"
server_key_version: "kuduclusterkey@0"
I20260926 14:57:27.306367 18721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "0ddf2f3ad0c24c8c80d3f15e16c3d1bf"
format_stamp: "Formatted at 2026-09-26 14:57:27 on dist-test-slave-f9xb"
server_key: "97550618d308b5ce67346ef8ca1c02f2"
server_key_iv: "9f10adbff3113c0909201ca286ae5ba5"
server_key_version: "kuduclusterkey@0"
I20260926 14:57:27.307988 18721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
Sep 26 14:57:27 dist-test-slave-f9xb krb5kdc[18027](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790434647, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:57:27.321776 18738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:57:27.321976 18721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:57:27.322064 18721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "0ddf2f3ad0c24c8c80d3f15e16c3d1bf"
format_stamp: "Formatted at 2026-09-26 14:57:27 on dist-test-slave-f9xb"
server_key: "97550618d308b5ce67346ef8ca1c02f2"
server_key_iv: "9f10adbff3113c0909201ca286ae5ba5"
server_key_version: "kuduclusterkey@0"
I20260926 14:57:27.322146 18721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:57:27.344374 18721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:57:27.346481 18721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:57:27.346650 18721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:57:27.346891 18721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:57:27.347265 18721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:57:27.347361 18721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:57:27.347425 18721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:57:27.347476 18721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:57:27.353570 18721 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:42787
I20260926 14:57:27.354274 18721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:57:27.356158 18851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:42787 every 8 connection(s)
I20260926 14:57:27.362001 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 18721
I20260926 14:57:27.362120 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:57:27.362305 7319 external_mini_cluster.cc:1428] Setting key bd7f2c32f9229fe44d1e44d2e03628d8
Sep 26 14:57:27 dist-test-slave-f9xb krb5kdc[18027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434647, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:57:27.378813 18717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.357051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:44581 (local address 127.7.37.254:42905)
0926 14:57:27.357096 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 14:57:27.357098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:57:27.360576 (+ 3478us) server_negotiation.cc:407] Connection header received
0926 14:57:27.360622 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:57:27.360623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:57:27.360639 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:57:27.360699 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:57:27.364052 (+ 3353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.364818 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.366681 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.366995 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.367033 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:57:27.375769 (+ 8736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:57:27.375771 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:57:27.375771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:57:27.375778 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:57:27.377425 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:57:27.377864 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:57:27.377866 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:57:27.377866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:57:27.377919 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:57:27.378148 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:57:27.378149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:57:27.378149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:57:27.378276 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:57:27.378316 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 14:57:27.378633 (+ 317us) server_negotiation.cc:299] Negotiation successful
0926 14:57:27.378714 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 14:57:27.379473 18854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.357046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42905 (local address 127.7.37.194:44581)
0926 14:57:27.358144 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0926 14:57:27.358188 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 14:57:27.358777 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:57:27.361348 (+ 2571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:57:27.361352 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:57:27.361574 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:57:27.363946 (+ 2372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:57:27.363952 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.365328 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.365328 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:57:27.366559 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:57:27.366562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.372273 (+ 5711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.372273 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:57:27.372416 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:57:27.374066 (+ 1650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:57:27.374075 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:57:27.375643 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:57:27.377517 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:57:27.377522 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:57:27.377524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:57:27.377785 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:57:27.377979 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:57:27.377980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:57:27.377980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:57:27.378089 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:57:27.378359 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:57:27.378364 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:57:27.378504 (+ 140us) client_negotiation.cc:769] Sending connection context
0926 14:57:27.379114 (+ 610us) client_negotiation.cc:240] Negotiation successful
0926 14:57:27.379262 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1058,"thread_start_us":75,"threads_started":1}
I20260926 14:57:27.380038 18852 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42905
I20260926 14:57:27.380117 18852 heartbeater.cc:464] Registering TS with master...
I20260926 14:57:27.380280 18852 heartbeater.cc:511] Master 127.7.37.254:42905 requested a full tablet report, sending...
I20260926 14:57:27.380551 18503 ts_manager.cc:194] Registered new tserver with Master: 0ddf2f3ad0c24c8c80d3f15e16c3d1bf (127.7.37.194:42787)
I20260926 14:57:27.381000 18503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:44581
WARNING: no policy specified for kudu/127.7.37.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
I20260926 14:57:27.398969 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.37.212:46253/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42905
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:39373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:57:27.518021 18857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:57:27.518359 18857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:57:27.518445 18857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:57:27.520081 18857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:57:27.520179 18857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:57:27.520273 18857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:57:27.522294 18857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:39373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.37.212:46253/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:57:27.522769 18857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:57:27.523046 18857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:57:27.529217 18864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:57:27.529623 18863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:57:27.529929 18866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:57:27.534413 18857 server_base.cc:1034] running on GCE node
I20260926 14:57:27.534724 18857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:57:27.534956 18857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:57:27.550379 18857 hybrid_clock.cc:648] HybridClock initialized: now 1790434647550350 us; error 42 us; skew 500 ppm
Sep 26 14:57:27 dist-test-slave-f9xb krb5kdc[18027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434647, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:57:27.553382 18857 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:57:27.553828 18857 webserver.cc:460] Webserver started at http://127.7.37.195:42807/ using document root <none> and password file <none>
I20260926 14:57:27.554039 18857 fs_manager.cc:359] Metadata directory not provided
I20260926 14:57:27.554082 18857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:57:27.554204 18857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 14:57:27 dist-test-slave-f9xb krb5kdc[18027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434647, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
I20260926 14:57:27.566385 18857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "f9b0ab868b574bfcac471ee8eea352da"
format_stamp: "Formatted at 2026-09-26 14:57:27 on dist-test-slave-f9xb"
server_key: "4fbaf3751791a6d0096af3f15344fedb"
server_key_iv: "3721b960d660f0dd8070f0d3d9cf52b9"
server_key_version: "kuduclusterkey@0"
I20260926 14:57:27.566774 18857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "f9b0ab868b574bfcac471ee8eea352da"
format_stamp: "Formatted at 2026-09-26 14:57:27 on dist-test-slave-f9xb"
server_key: "4fbaf3751791a6d0096af3f15344fedb"
server_key_iv: "3721b960d660f0dd8070f0d3d9cf52b9"
server_key_version: "kuduclusterkey@0"
I20260926 14:57:27.568503 18857 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 14:57:27.596276 18873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:57:27.596879 18857 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.000s
I20260926 14:57:27.596982 18857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "f9b0ab868b574bfcac471ee8eea352da"
format_stamp: "Formatted at 2026-09-26 14:57:27 on dist-test-slave-f9xb"
server_key: "4fbaf3751791a6d0096af3f15344fedb"
server_key_iv: "3721b960d660f0dd8070f0d3d9cf52b9"
server_key_version: "kuduclusterkey@0"
I20260926 14:57:27.597069 18857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:57:27.616575 18857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:57:27.620486 18857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:57:27.620617 18857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:57:27.620842 18857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:57:27.621201 18857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:57:27.621254 18857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:57:27.621280 18857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:57:27.621295 18857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:57:27.628635 18857 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:36059
I20260926 14:57:27.629029 18857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:57:27.630609 18986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:36059 every 8 connection(s)
Sep 26 14:57:27 dist-test-slave-f9xb krb5kdc[18027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434647, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:57:27.637045 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 18857
I20260926 14:57:27.637153 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:57:27.637327 7319 external_mini_cluster.cc:1428] Setting key 6590d95f3dbb8cfa2340d9db796ed4f1
I20260926 14:57:27.639547 18717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.629723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:39863 (local address 127.7.37.254:42905)
0926 14:57:27.629764 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 14:57:27.629767 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:57:27.629997 (+ 230us) server_negotiation.cc:407] Connection header received
0926 14:57:27.630751 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:57:27.630753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:57:27.630766 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:57:27.630796 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:57:27.631603 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.632373 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.633783 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.634071 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.634143 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:57:27.636422 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:57:27.636423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:57:27.636423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:57:27.636431 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:57:27.638002 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:57:27.638552 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:57:27.638553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:57:27.638554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:57:27.638614 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:57:27.638886 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:57:27.638887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:57:27.638887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:57:27.639022 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:57:27.639088 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 14:57:27.639382 (+ 294us) server_negotiation.cc:299] Negotiation successful
0926 14:57:27.639456 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:57:27.639693 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.629714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42905 (local address 127.7.37.195:39863)
0926 14:57:27.629867 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 14:57:27.629911 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 14:57:27.630569 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:57:27.630971 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:57:27.630975 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:57:27.631218 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:57:27.631508 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:57:27.631515 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.632485 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.632486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:57:27.633694 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:57:27.633697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.634150 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.634151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:57:27.634282 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:57:27.634791 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:57:27.634800 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:57:27.636342 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:57:27.638081 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:57:27.638087 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:57:27.638089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:57:27.638464 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:57:27.638703 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:57:27.638703 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:57:27.638704 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:57:27.638822 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:57:27.639087 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:57:27.639091 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:57:27.639249 (+ 158us) client_negotiation.cc:769] Sending connection context
0926 14:57:27.639358 (+ 109us) client_negotiation.cc:240] Negotiation successful
0926 14:57:27.639494 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260926 14:57:27.640609 18987 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42905
I20260926 14:57:27.640705 18987 heartbeater.cc:464] Registering TS with master...
I20260926 14:57:27.640911 18987 heartbeater.cc:511] Master 127.7.37.254:42905 requested a full tablet report, sending...
I20260926 14:57:27.641304 18503 ts_manager.cc:194] Registered new tserver with Master: f9b0ab868b574bfcac471ee8eea352da (127.7.37.195:36059)
I20260926 14:57:27.641752 18503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:39863
I20260926 14:57:27.656353 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 14:57:27.667685 18717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.660049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48720 (local address 127.7.37.254:42905)
0926 14:57:27.660087 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 14:57:27.660089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:57:27.660344 (+ 255us) server_negotiation.cc:407] Connection header received
0926 14:57:27.660430 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:57:27.660431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:57:27.660442 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:57:27.660469 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:57:27.661195 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.661981 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.663075 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.663371 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.663406 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:57:27.664866 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:57:27.664868 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:57:27.664868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:57:27.664875 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:57:27.666421 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:57:27.666795 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:57:27.666796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:57:27.666797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:57:27.666849 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:57:27.667029 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:57:27.667030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:57:27.667030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:57:27.667144 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:57:27.667211 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 14:57:27.667461 (+ 250us) server_negotiation.cc:299] Negotiation successful
0926 14:57:27.667569 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 14:57:27.669342 18503 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:57:27.671464 18503 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:57:27.683404 19000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.678021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:40509 (local address 127.0.0.1:47932)
0926 14:57:27.679111 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0926 14:57:27.679149 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 14:57:27.679209 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:57:27.679509 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:57:27.679511 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:57:27.679514 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:57:27.679573 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:57:27.679574 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.680948 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.680948 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:57:27.682188 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:57:27.682191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.683133 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.683134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:57:27.683217 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:57:27.683229 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 14:57:27.683288 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 14:57:27.683333 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1065,"thread_start_us":58,"threads_started":1}
I20260926 14:57:27.683917 19001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.678233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47932 (local address 127.7.37.193:40509)
0926 14:57:27.679324 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0926 14:57:27.679328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:57:27.679337 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:57:27.679366 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:57:27.679368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:57:27.679383 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:57:27.679443 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:57:27.679799 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.680861 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.682276 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.682986 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.683049 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:57:27.683102 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 14:57:27.683803 (+ 701us) server_negotiation.cc:299] Negotiation successful
0926 14:57:27.683847 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1061,"thread_start_us":47,"threads_started":1}
I20260926 14:57:27.684617 18648 tablet_service.cc:1467] Processing CreateTablet for tablet 1981931241df4de89705d2cc28e71318 (DEFAULT_TABLE table=test-table [id=ee99262c06a442319bb4125e1beca87e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:57:27.684949 18648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1981931241df4de89705d2cc28e71318. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:57:27.690610 19002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.678397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:36059 (local address 127.0.0.1:46388)
0926 14:57:27.679658 (+ 1261us) negotiation.cc:107] Waiting for socket to connect
0926 14:57:27.679667 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:57:27.679697 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:57:27.683789 (+ 4092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:57:27.683790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:57:27.683792 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:57:27.683833 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:57:27.683835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.688397 (+ 4562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.688398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:57:27.689466 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:57:27.689469 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.690414 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.690415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:57:27.690474 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:57:27.690479 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 14:57:27.690506 (+ 27us) client_negotiation.cc:240] Negotiation successful
0926 14:57:27.690540 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1247,"thread_start_us":52,"threads_started":1}
I20260926 14:57:27.691118 19004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.678773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46388 (local address 127.7.37.195:36059)
0926 14:57:27.683580 (+ 4807us) server_negotiation.cc:206] Beginning negotiation
0926 14:57:27.683584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:57:27.683617 (+ 33us) server_negotiation.cc:407] Connection header received
0926 14:57:27.683665 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:57:27.683667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:57:27.683681 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:57:27.685938 (+ 2257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:57:27.686011 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.687072 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.689578 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.690306 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.690906 (+ 600us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:57:27.690985 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 14:57:27.691010 (+ 25us) server_negotiation.cc:299] Negotiation successful
0926 14:57:27.691045 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4770,"thread_start_us":71,"threads_started":1}
I20260926 14:57:27.691843 18902 tablet_service.cc:1467] Processing CreateTablet for tablet 1981931241df4de89705d2cc28e71318 (DEFAULT_TABLE table=test-table [id=ee99262c06a442319bb4125e1beca87e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:57:27.692152 18902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1981931241df4de89705d2cc28e71318. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:57:27.695446 19007 tablet_bootstrap.cc:492] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da: Bootstrap starting.
I20260926 14:57:27.696491 19006 tablet_bootstrap.cc:492] T 1981931241df4de89705d2cc28e71318 P 60024aff442d4cbd88ff173839631ca3: Bootstrap starting.
I20260926 14:57:27.696535 19007 tablet_bootstrap.cc:654] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da: Neither blocks nor log segments found. Creating new log.
I20260926 14:57:27.696805 19007 log.cc:824] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da: Log is configured to *not* fsync() on all Append() calls
I20260926 14:57:27.697187 19006 tablet_bootstrap.cc:654] T 1981931241df4de89705d2cc28e71318 P 60024aff442d4cbd88ff173839631ca3: Neither blocks nor log segments found. Creating new log.
I20260926 14:57:27.697579 19003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.678514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:42787 (local address 127.0.0.1:42498)
0926 14:57:27.683188 (+ 4674us) negotiation.cc:107] Waiting for socket to connect
0926 14:57:27.683197 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:57:27.683237 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:57:27.693403 (+ 10166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:57:27.693403 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:57:27.693405 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:57:27.693439 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:57:27.693440 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.695342 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.695343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:57:27.696471 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:57:27.696474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.697379 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.697380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:57:27.697441 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:57:27.697445 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:57:27.697473 (+ 28us) client_negotiation.cc:240] Negotiation successful
0926 14:57:27.697509 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4659,"thread_start_us":52,"threads_started":1}
I20260926 14:57:27.697584 19006 log.cc:824] T 1981931241df4de89705d2cc28e71318 P 60024aff442d4cbd88ff173839631ca3: Log is configured to *not* fsync() on all Append() calls
I20260926 14:57:27.698195 19005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.678919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42498 (local address 127.7.37.194:42787)
0926 14:57:27.693069 (+ 14150us) server_negotiation.cc:206] Beginning negotiation
0926 14:57:27.693073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:57:27.693084 (+ 11us) server_negotiation.cc:407] Connection header received
0926 14:57:27.693163 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:57:27.693166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:57:27.693180 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:57:27.693251 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:57:27.693512 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.694586 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.696584 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.697284 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.697900 (+ 616us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:57:27.698029 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 14:57:27.698064 (+ 35us) server_negotiation.cc:299] Negotiation successful
0926 14:57:27.698107 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14112,"thread_start_us":61,"threads_started":1}
I20260926 14:57:27.698741 19006 tablet_bootstrap.cc:492] T 1981931241df4de89705d2cc28e71318 P 60024aff442d4cbd88ff173839631ca3: No bootstrap required, opened a new log
I20260926 14:57:27.698800 19006 ts_tablet_manager.cc:1397] T 1981931241df4de89705d2cc28e71318 P 60024aff442d4cbd88ff173839631ca3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:57:27.699131 18786 tablet_service.cc:1467] Processing CreateTablet for tablet 1981931241df4de89705d2cc28e71318 (DEFAULT_TABLE table=test-table [id=ee99262c06a442319bb4125e1beca87e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:57:27.699426 18786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1981931241df4de89705d2cc28e71318. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:57:27.701056 19006 raft_consensus.cc:348] T 1981931241df4de89705d2cc28e71318 P 60024aff442d4cbd88ff173839631ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9b0ab868b574bfcac471ee8eea352da" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36059 } } peers { permanent_uuid: "60024aff442d4cbd88ff173839631ca3" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40509 } } peers { permanent_uuid: "0ddf2f3ad0c24c8c80d3f15e16c3d1bf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42787 } }
I20260926 14:57:27.701215 19006 raft_consensus.cc:374] T 1981931241df4de89705d2cc28e71318 P 60024aff442d4cbd88ff173839631ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:57:27.701246 19006 raft_consensus.cc:729] T 1981931241df4de89705d2cc28e71318 P 60024aff442d4cbd88ff173839631ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60024aff442d4cbd88ff173839631ca3, State: Initialized, Role: FOLLOWER
I20260926 14:57:27.701349 19006 consensus_queue.cc:260] T 1981931241df4de89705d2cc28e71318 P 60024aff442d4cbd88ff173839631ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9b0ab868b574bfcac471ee8eea352da" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36059 } } peers { permanent_uuid: "60024aff442d4cbd88ff173839631ca3" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40509 } } peers { permanent_uuid: "0ddf2f3ad0c24c8c80d3f15e16c3d1bf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42787 } }
I20260926 14:57:27.701567 19006 ts_tablet_manager.cc:1428] T 1981931241df4de89705d2cc28e71318 P 60024aff442d4cbd88ff173839631ca3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 14:57:27.703176 18715 heartbeater.cc:503] Master 127.7.37.254:42905 was elected leader, sending a full tablet report...
I20260926 14:57:27.704421 19011 tablet_bootstrap.cc:492] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf: Bootstrap starting.
I20260926 14:57:27.705292 19011 tablet_bootstrap.cc:654] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf: Neither blocks nor log segments found. Creating new log.
I20260926 14:57:27.705603 19011 log.cc:824] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf: Log is configured to *not* fsync() on all Append() calls
I20260926 14:57:27.706539 19011 tablet_bootstrap.cc:492] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf: No bootstrap required, opened a new log
I20260926 14:57:27.706600 19011 ts_tablet_manager.cc:1397] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:57:27.708433 19011 raft_consensus.cc:348] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9b0ab868b574bfcac471ee8eea352da" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36059 } } peers { permanent_uuid: "60024aff442d4cbd88ff173839631ca3" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40509 } } peers { permanent_uuid: "0ddf2f3ad0c24c8c80d3f15e16c3d1bf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42787 } }
I20260926 14:57:27.708564 19011 raft_consensus.cc:374] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:57:27.708591 19011 raft_consensus.cc:729] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ddf2f3ad0c24c8c80d3f15e16c3d1bf, State: Initialized, Role: FOLLOWER
I20260926 14:57:27.708685 19011 consensus_queue.cc:260] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9b0ab868b574bfcac471ee8eea352da" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36059 } } peers { permanent_uuid: "60024aff442d4cbd88ff173839631ca3" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40509 } } peers { permanent_uuid: "0ddf2f3ad0c24c8c80d3f15e16c3d1bf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42787 } }
I20260926 14:57:27.709219 19011 ts_tablet_manager.cc:1428] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 14:57:27.709333 18852 heartbeater.cc:503] Master 127.7.37.254:42905 was elected leader, sending a full tablet report...
I20260926 14:57:27.711125 19007 tablet_bootstrap.cc:492] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da: No bootstrap required, opened a new log
I20260926 14:57:27.711191 19007 ts_tablet_manager.cc:1397] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20260926 14:57:27.713014 19007 raft_consensus.cc:348] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9b0ab868b574bfcac471ee8eea352da" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36059 } } peers { permanent_uuid: "60024aff442d4cbd88ff173839631ca3" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40509 } } peers { permanent_uuid: "0ddf2f3ad0c24c8c80d3f15e16c3d1bf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42787 } }
I20260926 14:57:27.713165 19007 raft_consensus.cc:374] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:57:27.713199 19007 raft_consensus.cc:729] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9b0ab868b574bfcac471ee8eea352da, State: Initialized, Role: FOLLOWER
I20260926 14:57:27.713296 19007 consensus_queue.cc:260] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9b0ab868b574bfcac471ee8eea352da" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36059 } } peers { permanent_uuid: "60024aff442d4cbd88ff173839631ca3" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40509 } } peers { permanent_uuid: "0ddf2f3ad0c24c8c80d3f15e16c3d1bf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42787 } }
I20260926 14:57:27.713532 19007 ts_tablet_manager.cc:1428] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:57:27.713744 18987 heartbeater.cc:503] Master 127.7.37.254:42905 was elected leader, sending a full tablet report...
Sep 26 14:57:27 dist-test-slave-f9xb krb5kdc[18027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434647, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
I20260926 14:57:27.843768 19013 raft_consensus.cc:482] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:57:27.843904 19013 raft_consensus.cc:504] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9b0ab868b574bfcac471ee8eea352da" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36059 } } peers { permanent_uuid: "60024aff442d4cbd88ff173839631ca3" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40509 } } peers { permanent_uuid: "0ddf2f3ad0c24c8c80d3f15e16c3d1bf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42787 } }
I20260926 14:57:27.844326 19013 leader_election.cc:288] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9b0ab868b574bfcac471ee8eea352da (127.7.37.195:36059), 60024aff442d4cbd88ff173839631ca3 (127.7.37.193:40509)
I20260926 14:57:27.847879 18854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.844451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:36059 (local address 127.7.37.194:41965)
0926 14:57:27.844542 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0926 14:57:27.844553 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:57:27.844590 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:57:27.844892 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:57:27.844893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:57:27.844894 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:57:27.844960 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:57:27.844961 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.845836 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.845837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:57:27.846935 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:57:27.846938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.847627 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.847628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:57:27.847704 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:57:27.847707 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:57:27.847771 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 14:57:27.847811 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260926 14:57:27.848160 19001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.844725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:51305 (local address 127.7.37.193:40509)
0926 14:57:27.844936 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 14:57:27.844938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:57:27.844945 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:57:27.844966 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:57:27.844967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:57:27.844976 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:57:27.845017 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:57:27.845298 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.846005 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.847198 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.847788 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.847835 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:57:27.847842 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 14:57:27.848053 (+ 211us) server_negotiation.cc:299] Negotiation successful
0926 14:57:27.848096 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
I20260926 14:57:27.848277 19004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.844672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:41965 (local address 127.7.37.195:36059)
0926 14:57:27.844774 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0926 14:57:27.844776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:57:27.844784 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:57:27.844812 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:57:27.844813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:57:27.844824 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:57:27.844884 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:57:27.845030 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.845735 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.847035 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.847533 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.847591 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:57:27.847600 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 14:57:27.848181 (+ 581us) server_negotiation.cc:299] Negotiation successful
0926 14:57:27.848220 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 14:57:27.848582 19015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.844520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:40509 (local address 127.7.37.194:51305)
0926 14:57:27.844728 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 14:57:27.844736 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:57:27.844771 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:57:27.845125 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:57:27.845126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:57:27.845128 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:57:27.845207 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:57:27.845209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.846099 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.846100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:57:27.847101 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:57:27.847104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.847910 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.847911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:57:27.847978 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:57:27.847981 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:57:27.848460 (+ 479us) client_negotiation.cc:240] Negotiation successful
0926 14:57:27.848500 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":79,"threads_started":1}
I20260926 14:57:27.848529 18941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1981931241df4de89705d2cc28e71318" candidate_uuid: "0ddf2f3ad0c24c8c80d3f15e16c3d1bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9b0ab868b574bfcac471ee8eea352da" is_pre_election: true
I20260926 14:57:27.848682 18941 raft_consensus.cc:2454] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ddf2f3ad0c24c8c80d3f15e16c3d1bf in term 0.
I20260926 14:57:27.849284 18740 leader_election.cc:302] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ddf2f3ad0c24c8c80d3f15e16c3d1bf, f9b0ab868b574bfcac471ee8eea352da; no voters:
I20260926 14:57:27.849349 18666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1981931241df4de89705d2cc28e71318" candidate_uuid: "0ddf2f3ad0c24c8c80d3f15e16c3d1bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60024aff442d4cbd88ff173839631ca3" is_pre_election: true
I20260926 14:57:27.849488 18666 raft_consensus.cc:2454] T 1981931241df4de89705d2cc28e71318 P 60024aff442d4cbd88ff173839631ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ddf2f3ad0c24c8c80d3f15e16c3d1bf in term 0.
I20260926 14:57:27.849588 19013 raft_consensus.cc:2781] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:57:27.849638 19013 raft_consensus.cc:482] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:57:27.849659 19013 raft_consensus.cc:3037] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:57:27.850483 19013 raft_consensus.cc:504] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9b0ab868b574bfcac471ee8eea352da" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36059 } } peers { permanent_uuid: "60024aff442d4cbd88ff173839631ca3" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40509 } } peers { permanent_uuid: "0ddf2f3ad0c24c8c80d3f15e16c3d1bf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42787 } }
I20260926 14:57:27.850608 19013 leader_election.cc:288] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [CANDIDATE]: Term 1 election: Requested vote from peers f9b0ab868b574bfcac471ee8eea352da (127.7.37.195:36059), 60024aff442d4cbd88ff173839631ca3 (127.7.37.193:40509)
I20260926 14:57:27.850800 18941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1981931241df4de89705d2cc28e71318" candidate_uuid: "0ddf2f3ad0c24c8c80d3f15e16c3d1bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9b0ab868b574bfcac471ee8eea352da"
I20260926 14:57:27.850875 18941 raft_consensus.cc:3037] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:57:27.850893 18666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1981931241df4de89705d2cc28e71318" candidate_uuid: "0ddf2f3ad0c24c8c80d3f15e16c3d1bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60024aff442d4cbd88ff173839631ca3"
I20260926 14:57:27.850960 18666 raft_consensus.cc:3037] T 1981931241df4de89705d2cc28e71318 P 60024aff442d4cbd88ff173839631ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:57:27.851817 18941 raft_consensus.cc:2454] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ddf2f3ad0c24c8c80d3f15e16c3d1bf in term 1.
I20260926 14:57:27.851819 18666 raft_consensus.cc:2454] T 1981931241df4de89705d2cc28e71318 P 60024aff442d4cbd88ff173839631ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ddf2f3ad0c24c8c80d3f15e16c3d1bf in term 1.
I20260926 14:57:27.851989 18740 leader_election.cc:302] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ddf2f3ad0c24c8c80d3f15e16c3d1bf, f9b0ab868b574bfcac471ee8eea352da; no voters:
I20260926 14:57:27.852083 19013 raft_consensus.cc:2781] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:57:27.852246 19013 raft_consensus.cc:686] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [term 1 LEADER]: Becoming Leader. State: Replica: 0ddf2f3ad0c24c8c80d3f15e16c3d1bf, State: Running, Role: LEADER
I20260926 14:57:27.852332 19013 consensus_queue.cc:237] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9b0ab868b574bfcac471ee8eea352da" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36059 } } peers { permanent_uuid: "60024aff442d4cbd88ff173839631ca3" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40509 } } peers { permanent_uuid: "0ddf2f3ad0c24c8c80d3f15e16c3d1bf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42787 } }
I20260926 14:57:27.853156 18502 catalog_manager.cc:5495] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ddf2f3ad0c24c8c80d3f15e16c3d1bf (127.7.37.194). New cstate: current_term: 1 leader_uuid: "0ddf2f3ad0c24c8c80d3f15e16c3d1bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9b0ab868b574bfcac471ee8eea352da" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60024aff442d4cbd88ff173839631ca3" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ddf2f3ad0c24c8c80d3f15e16c3d1bf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42787 } health_report { overall_health: HEALTHY } } }
W20260926 14:57:27.857605 18853 tablet.cc:2369] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:57:27.862571 19005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.859603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42500 (local address 127.7.37.194:42787)
0926 14:57:27.859644 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 14:57:27.859646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:57:27.859654 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:57:27.859711 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:57:27.859712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:57:27.859727 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:57:27.859808 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:57:27.859960 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.860618 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.861645 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:57:27.861908 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:57:27.861960 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:57:27.862128 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:57:27.862246 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:57:27.862313 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 14:57:27.862498 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 14:57:27.862515 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 14:57:27.864226 18941 raft_consensus.cc:1261] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da [term 1 FOLLOWER]: Refusing update from remote peer 0ddf2f3ad0c24c8c80d3f15e16c3d1bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:57:27.864234 18666 raft_consensus.cc:1261] T 1981931241df4de89705d2cc28e71318 P 60024aff442d4cbd88ff173839631ca3 [term 1 FOLLOWER]: Refusing update from remote peer 0ddf2f3ad0c24c8c80d3f15e16c3d1bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:57:27.864490 19013 consensus_queue.cc:1035] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "60024aff442d4cbd88ff173839631ca3" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:57:27.864603 19016 consensus_queue.cc:1035] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9b0ab868b574bfcac471ee8eea352da" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 36059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:57:27.866981 19019 mvcc.cc:205] Tried to move back new op lower bound from 7333620317649518592 to 7333620317603770368. Current Snapshot: MvccSnapshot[applied={T|T < 7333620317649518592}]
I20260926 14:57:27.867976 19020 mvcc.cc:205] Tried to move back new op lower bound from 7333620317649518592 to 7333620317603770368. Current Snapshot: MvccSnapshot[applied={T|T < 7333620317649518592}]
I20260926 14:57:27.868590 19021 mvcc.cc:205] Tried to move back new op lower bound from 7333620317649518592 to 7333620317603770368. Current Snapshot: MvccSnapshot[applied={T|T < 7333620317649518592}]
I20260926 14:57:27.870450 18503 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:57:27.870522 18503 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:57:27.871129 18503 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52cae4e38bf140488e8c3b3ddcb9bdd5: Sending DeleteTablet for 3 replicas of tablet 1981931241df4de89705d2cc28e71318
I20260926 14:57:27.872196 18648 tablet_service.cc:1514] Processing DeleteTablet for tablet 1981931241df4de89705d2cc28e71318 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:57:27 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:47932
I20260926 14:57:27.872603 18902 tablet_service.cc:1514] Processing DeleteTablet for tablet 1981931241df4de89705d2cc28e71318 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:57:27 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:46388
I20260926 14:57:27.872777 18786 tablet_service.cc:1514] Processing DeleteTablet for tablet 1981931241df4de89705d2cc28e71318 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:57:27 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:42498
I20260926 14:57:27.873281 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 18584
I20260926 14:57:27.876269 19029 tablet_replica.cc:330] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da: stopping tablet replica
I20260926 14:57:27.876385 19030 tablet_replica.cc:330] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf: stopping tablet replica
I20260926 14:57:27.876488 19030 raft_consensus.cc:2229] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:57:27.876581 19029 raft_consensus.cc:2229] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:57:27.876735 19030 raft_consensus.cc:2258] T 1981931241df4de89705d2cc28e71318 P 0ddf2f3ad0c24c8c80d3f15e16c3d1bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:57:27.876847 19029 raft_consensus.cc:2258] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 14:57:27.880035 18487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.37.193:40509 (error 108)
I20260926 14:57:27.880502 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 18721
I20260926 14:57:27.880520 19003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.880398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:40509 (local address 127.0.0.1:47946)
0926 14:57:27.880450 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 14:57:27.880490 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.193:40509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260926 14:57:27.881218 19029 ts_tablet_manager.cc:1905] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 14:57:27.881470 18487 catalog_manager.cc:4584] TS 60024aff442d4cbd88ff173839631ca3 (127.7.37.193:40509): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1981931241df4de89705d2cc28e71318: Network error: Client connection negotiation failed: client connection to 127.7.37.193:40509: connect: Connection refused (error 111)
I20260926 14:57:27.882735 19029 ts_tablet_manager.cc:1918] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:57:27.882802 19029 log.cc:1196] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/1981931241df4de89705d2cc28e71318
I20260926 14:57:27.883330 19029 ts_tablet_manager.cc:1939] T 1981931241df4de89705d2cc28e71318 P f9b0ab868b574bfcac471ee8eea352da: Deleting consensus metadata
I20260926 14:57:27.883692 18488 catalog_manager.cc:4841] TS f9b0ab868b574bfcac471ee8eea352da (127.7.37.195:36059): tablet 1981931241df4de89705d2cc28e71318 (table test-table [id=ee99262c06a442319bb4125e1beca87e]) successfully deleted
I20260926 14:57:27.891091 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 18857
I20260926 14:57:27.891170 19003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:57:27.891073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:42787 (local address 127.0.0.1:42516)
0926 14:57:27.891116 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 14:57:27.891141 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.194:42787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 14:57:27.895895 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 18471
I20260926 14:57:27.901669 7319 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 14:57:28.235193 7319 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 14:57:28.235291 7319 mini_ranger.cc:66] Stopping Ranger...
I20260926 14:57:28.568720 7319 mini_ranger.cc:68] Stopped Ranger
2026-09-26 14:57:28.568 UTC [18057] LOG: received smart shutdown request
2026-09-26 14:57:28.572 UTC [18057] LOG: background worker "logical replication launcher" (PID 18066) exited with exit code 1
2026-09-26 14:57:28.572 UTC [18061] LOG: shutting down
2026-09-26 14:57:28.737 UTC [18057] LOG: database system is shut down
2026-09-26T14:57:28Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (40897 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:57:28 dist-test-slave-f9xb krb5kdc[19040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:57:28 dist-test-slave-f9xb krb5kdc[19040](info): set up 2 sockets
Sep 26 14:57:28 dist-test-slave-f9xb krb5kdc[19040](info): commencing operation
krb5kdc: starting...
W20260926 14:57:29.775290 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:57:29 dist-test-slave-f9xb krb5kdc[19040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:57:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:57:29Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.37.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.37.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.37.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/rangeradmin_127.7.37.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.37.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/rangeradmin_127.7.37.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.37.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.37.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.37.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/rangerlookup_127.7.37.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.37.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/rangerlookup_127.7.37.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.37.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.37.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.37.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/HTTP_127.7.37.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.37.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/HTTP_127.7.37.212@KRBTEST.COM.keytab.
I20260926 14:57:29.912141 7319 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskrMzbz4/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/postgres -l logfile start
2026-09-26 14:57:32.118 UTC [19070] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 14:57:32.118 UTC [19070] LOG: listening on IPv4 address "127.7.37.212", port 51763
2026-09-26 14:57:32.121 UTC [19070] LOG: listening on Unix socket "/tmp/.s.PGSQL.51763"
2026-09-26 14:57:32.140 UTC [19073] LOG: database system was shut down at 2026-09-26 14:57:30 UTC
2026-09-26 14:57:32.142 UTC [19070] LOG: database system is ready to accept connections
I20260926 14:57:33.003266 7319 mini_postgres.cc:96] Postgres bound to 51763
2026-09-26 14:57:33.008 UTC [19081] FATAL: database "slave" does not exist
127.7.37.212:51763 - accepting connections
I20260926 14:57:33.008963 7319 mini_ranger.cc:161] Starting Ranger...
I20260926 14:57:33.022063 7319 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 14:57:33.431357 7319 mini_ranger.cc:87] Created ranger Postgres database
I20260926 14:57:33.431449 7319 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ranger-admin
2026-09-26 14:57:33,680 [I] DB FLAVOR :POSTGRES
2026-09-26 14:57:33,681 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 14:57:33,681 [I] Checking connection..
2026-09-26 14:57:33,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 14:57:33,905 [I] Checking connection passed.
2026-09-26 14:57:33,905 [I] --------- Verifying version history table ---------
2026-09-26 14:57:33,905 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 14:57:34,128 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 14:57:34,128 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 14:57:34,360 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 14:57:34,360 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 14:57:34,361 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 14:57:34.578 UTC [19145] WARNING: there is no transaction in progress
2026-09-26 14:57:34,583 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 14:57:34,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 14:57:34,823 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 14:57:34,823 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 14:57:34,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 14:57:35,040 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 14:57:35,268 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-f9xb.c.gcp-upstream.internal','N') ;"
2026-09-26 14:57:35,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 14:57:35,704 [I] Table x_portal_user does not exist in database ranger
2026-09-26 14:57:35,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 14:57:35,914 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 14:57:35,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 14:57:36,154 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 14:57:36,375 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 14:57:36,375 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 14:57:37.215 UTC [19253] WARNING: there is no transaction in progress
2026-09-26 14:57:37.230 UTC [19253] WARNING: there is no transaction in progress
2026-09-26 14:57:37.244 UTC [19253] WARNING: there is no transaction in progress
2026-09-26 14:57:37.258 UTC [19253] WARNING: there is no transaction in progress
2026-09-26 14:57:37.271 UTC [19253] WARNING: there is no transaction in progress
2026-09-26 14:57:37.288 UTC [19253] WARNING: there is no transaction in progress
2026-09-26 14:57:37.337 UTC [19253] WARNING: there is no transaction in progress
2026-09-26 14:57:37.343 UTC [19253] WARNING: there is no transaction in progress
2026-09-26 14:57:37.356 UTC [19253] WARNING: there is no transaction in progress
2026-09-26 14:57:37.365 UTC [19253] WARNING: there is no transaction in progress
2026-09-26 14:57:37.374 UTC [19253] WARNING: there is no transaction in progress
2026-09-26 14:57:37.387 UTC [19253] WARNING: there is no transaction in progress
2026-09-26 14:57:37.395 UTC [19253] WARNING: there is no transaction in progress
2026-09-26 14:57:37.401 UTC [19253] WARNING: there is no transaction in progress
2026-09-26 14:57:37.406 UTC [19253] WARNING: there is no transaction in progress
2026-09-26 14:57:37.412 UTC [19253] WARNING: there is no transaction in progress
2026-09-26 14:57:37,962 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 14:57:37,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 14:57:38,187 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 14:57:38,188 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 14:57:38,434 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 14:57:38,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 14:57:38,677 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 14:57:38,678 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 14:57:38,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 14:57:39,125 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-f9xb.c.gcp-upstream.internal';"
2026-09-26 14:57:39,341 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 14:57:39,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 14:57:39,578 [I] DB_PATCHES have already been applied
I20260926 14:57:39.583930 7319 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ranger-admin:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskrMzbz4/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 14:57:39.584005 7319 mini_ranger.cc:193] Using host: 127.7.37.212
I20260926 14:57:39.586756 7319 mini_ranger.cc:227] Ranger admin URL: http://127.7.37.212:56675
Sep 26, 2026 2:57:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 2:57:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 2:57:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 2:57:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 2:57:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 2:57:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 2:57:40 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 14:57:40.589807 7319 mini_ranger.cc:160] Time spent starting Ranger: real 7.581s user 0.000s sys 0.008s
Sep 26, 2026 2:57:58 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 2:57:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 26, 2026 2:57:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 2:57:58 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 2:57:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 2:57:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 2:57:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 2:57:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 2:57:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 2:57:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 2:57:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 2:57:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 2:57:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 2:57:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 2:57:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 2:57:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 2:57:58 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 2:57:59 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 14:57:59.690325 7319 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.37.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.37.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.37.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/rangerkms_127.7.37.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.37.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/rangerkms_127.7.37.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.37.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.37.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.37.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/HTTP_127.7.37.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.37.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/HTTP_127.7.37.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 14:57:59 dist-test-slave-f9xb krb5kdc[19040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434679, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 14:57:59.764941 7319 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 14:57:59.773133 7319 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 14:58:00.512512 7319 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 14:58:00.512598 7319 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ranger-kms
I20260926 14:58:00.512614 7319 mini_ranger_kms.cc:227] Using postgres at 127.7.37.212:51763
2026-09-26 14:58:00,654 [I] DB FLAVOR :POSTGRES
2026-09-26 14:58:00,654 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 14:58:00,654 [I] Checking connection
2026-09-26 14:58:00,654 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 14:58:00,882 [I] connection success
2026-09-26 14:58:00,883 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 14:58:00,883 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 14:58:00,883 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 14:58:01,098 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 14:58:01,098 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 14:58:01,098 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 14:58:01,098 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.37.212:51763/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 14:58:01,325 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 14:58:01.707125 7319 mini_ranger_kms.cc:310] Created kms service
I20260926 14:58:01.867136 7319 mini_ranger_kms.cc:326] Created kudu user
I20260926 14:58:01.904276 7319 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 14:58:01 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434679, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
I20260926 14:58:02.232903 7319 mini_ranger_kms.cc:384] Added ranger policy
I20260926 14:58:02.233011 7319 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ranger-kms:/tmp/dist-test-taskrMzbz4/build/release/bin/postgresql.jar:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskrMzbz4/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 14:58:02.233031 7319 mini_ranger_kms.cc:240] Using host: 127.7.37.212
I20260926 14:58:02.236048 7319 mini_ranger_kms.cc:276] Ranger KMS PID: 19433
I20260926 14:58:02.236191 7319 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.37.212:59435
Sep 26, 2026 2:58:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 2:58:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 2:58:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 2:58:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 2:58:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 2:58:03 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 14:58:04.278791 7319 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.514s user 0.005s sys 0.120s
I20260926 14:58:04.278889 7319 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 14:58:04.278934 7319 mini_ranger_kms.cc:401] 127.7.37.212:59435/kms/v1/keys
Sep 26 14:58:04 dist-test-slave-f9xb krb5kdc[19040](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790434684, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.37.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 14:58:04 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434684, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.37.212@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
Sep 26 14:58:05 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434684, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.37.212@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
Sep 26 14:58:07 dist-test-slave-f9xb krb5kdc[19040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:58:07.246388 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:42017
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:36605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:42017
--ranger_config_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.37.212:59435/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:58:07.321026 19484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:07.321200 19484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:07.321218 19484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:07.322285 19484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:58:07.322324 19484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:07.322336 19484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:07.322347 19484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:58:07.322357 19484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:58:07.323818 19484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:36605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.37.212:59435/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:42017
--ranger_config_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:42017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:07.324020 19484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:07.324208 19484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:07.328356 19492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:07.328504 19489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:07.328584 19484 server_base.cc:1034] running on GCE node
W20260926 14:58:07.328362 19490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:07.328840 19484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:07.329082 19484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:07.330209 19484 hybrid_clock.cc:648] HybridClock initialized: now 1790434687330184 us; error 44 us; skew 500 ppm
Sep 26 14:58:07 dist-test-slave-f9xb krb5kdc[19040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434687, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:07.332332 19484 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:58:07.332690 19484 webserver.cc:460] Webserver started at http://127.7.37.254:36817/ using document root <none> and password file <none>
I20260926 14:58:07.332847 19484 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:07.332888 19484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:07.332964 19484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 14:58:07 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434687, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
I20260926 14:58:07.374711 19484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "55ccc7022b4d46d99ff94743d6e6929b"
format_stamp: "Formatted at 2026-09-26 14:58:07 on dist-test-slave-f9xb"
server_key: "8f65cfa3e940e490345e32147a5309aa"
server_key_iv: "2e9fa392bb85effac86a3ab2aa406c1f"
server_key_version: "kuduclusterkey@0"
I20260926 14:58:07.375054 19484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "55ccc7022b4d46d99ff94743d6e6929b"
format_stamp: "Formatted at 2026-09-26 14:58:07 on dist-test-slave-f9xb"
server_key: "8f65cfa3e940e490345e32147a5309aa"
server_key_iv: "2e9fa392bb85effac86a3ab2aa406c1f"
server_key_version: "kuduclusterkey@0"
I20260926 14:58:07.376297 19484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 14:58:07.385772 19499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:07.385932 19484 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:07.386003 19484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "55ccc7022b4d46d99ff94743d6e6929b"
format_stamp: "Formatted at 2026-09-26 14:58:07 on dist-test-slave-f9xb"
server_key: "8f65cfa3e940e490345e32147a5309aa"
server_key_iv: "2e9fa392bb85effac86a3ab2aa406c1f"
server_key_version: "kuduclusterkey@0"
I20260926 14:58:07.386071 19484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:07.403823 19484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:07.405443 19484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:07.405561 19484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:07.409263 19484 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:42017
I20260926 14:58:07.409320 19551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:42017 every 8 connection(s)
I20260926 14:58:07.409601 19484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:58:07.410175 19552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:07.410601 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 19484
I20260926 14:58:07.410691 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:58:07.410867 7319 external_mini_cluster.cc:1428] Setting key a54fe589c36aceba1e74183e50792380
I20260926 14:58:07.412806 19552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b: Bootstrap starting.
I20260926 14:58:07.413337 19552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:07.413580 19552 log.cc:824] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:07.414168 19552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b: No bootstrap required, opened a new log
I20260926 14:58:07.416177 19552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ccc7022b4d46d99ff94743d6e6929b" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42017 } }
I20260926 14:58:07.416337 19552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:07.416417 19552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55ccc7022b4d46d99ff94743d6e6929b, State: Initialized, Role: FOLLOWER
I20260926 14:58:07.416545 19552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ccc7022b4d46d99ff94743d6e6929b" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42017 } }
I20260926 14:58:07.416630 19552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:58:07.416688 19552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:58:07.416741 19552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:07.417343 19552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ccc7022b4d46d99ff94743d6e6929b" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42017 } }
I20260926 14:58:07.417450 19552 leader_election.cc:302] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55ccc7022b4d46d99ff94743d6e6929b; no voters:
I20260926 14:58:07.417596 19552 leader_election.cc:288] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:58:07.417744 19558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:58:07.417829 19552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:58:07.417959 19558 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [term 1 LEADER]: Becoming Leader. State: Replica: 55ccc7022b4d46d99ff94743d6e6929b, State: Running, Role: LEADER
I20260926 14:58:07.418080 19558 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ccc7022b4d46d99ff94743d6e6929b" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42017 } }
Sep 26 14:58:07 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:07.418433 19558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55ccc7022b4d46d99ff94743d6e6929b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ccc7022b4d46d99ff94743d6e6929b" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42017 } } }
I20260926 14:58:07.418497 19558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [sys.catalog]: This master's current role is: LEADER
I20260926 14:58:07.418591 19558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55ccc7022b4d46d99ff94743d6e6929b. Latest consensus state: current_term: 1 leader_uuid: "55ccc7022b4d46d99ff94743d6e6929b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ccc7022b4d46d99ff94743d6e6929b" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42017 } } }
I20260926 14:58:07.418627 19558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [sys.catalog]: This master's current role is: LEADER
I20260926 14:58:07.418803 19566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:58:07.419127 19552 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 14:58:07.422592 19566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:58:07.429154 19566 catalog_manager.cc:1347] Generated new cluster ID: d1aba86e54184be982f2185e0c95239c
I20260926 14:58:07.429195 19566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:58:07.430195 19555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:07.411330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35428 (local address 127.7.37.254:42017)
0926 14:58:07.411488 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:07.411497 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:07.411525 (+ 28us) server_negotiation.cc:407] Connection header received
0926 14:58:07.412076 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:07.412084 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:07.412144 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:07.412282 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:07.412973 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:07.414376 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:07.415440 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:07.416057 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:07.416150 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:07.418757 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:07.418762 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:07.418762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:07.418773 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:07.428305 (+ 9532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:07.428697 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:07.428700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:07.428701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:07.428801 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:07.429027 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:07.429027 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:07.429028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:07.429321 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:07.429438 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:07.429842 (+ 404us) server_negotiation.cc:299] Negotiation successful
0926 14:58:07.429985 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260926 14:58:07.439522 19566 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:58:07.440047 19566 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:58:07.442358 19566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b: Generated new TSK 0
I20260926 14:58:07.442502 19566 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
I20260926 14:58:08.096354 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.37.212:59435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42017
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:36605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:58:08.203158 19597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:08.203334 19597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:08.203354 19597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:08.204528 19597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:08.204578 19597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:08.204617 19597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:58:08.206681 19597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:36605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.37.212:59435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:08.206889 19597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:08.207156 19597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:08.212966 19605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:08.213029 19603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:08.213194 19602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:08.213416 19597 server_base.cc:1034] running on GCE node
I20260926 14:58:08.213610 19597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:08.213840 19597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:08.214972 19597 hybrid_clock.cc:648] HybridClock initialized: now 1790434688214951 us; error 42 us; skew 500 ppm
Sep 26 14:58:08 dist-test-slave-f9xb krb5kdc[19040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434688, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:08.221222 19597 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:58:08.221736 19597 webserver.cc:460] Webserver started at http://127.7.37.193:36537/ using document root <none> and password file <none>
I20260926 14:58:08.221985 19597 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:08.222054 19597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:08.222183 19597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 14:58:08 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434688, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
I20260926 14:58:08.234234 19597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "74c8ab1db5784e6faab3b44de24368a5"
format_stamp: "Formatted at 2026-09-26 14:58:08 on dist-test-slave-f9xb"
server_key: "36603aae66d3f3bdd7765b5d170109ef"
server_key_iv: "097da602d8cbeb731b1626a0ced2c051"
server_key_version: "kuduclusterkey@0"
I20260926 14:58:08.234563 19597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "74c8ab1db5784e6faab3b44de24368a5"
format_stamp: "Formatted at 2026-09-26 14:58:08 on dist-test-slave-f9xb"
server_key: "36603aae66d3f3bdd7765b5d170109ef"
server_key_iv: "097da602d8cbeb731b1626a0ced2c051"
server_key_version: "kuduclusterkey@0"
I20260926 14:58:08.235814 19597 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:58:08.259830 19614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:08.259995 19597 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.001s sys 0.000s
I20260926 14:58:08.260068 19597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "74c8ab1db5784e6faab3b44de24368a5"
format_stamp: "Formatted at 2026-09-26 14:58:08 on dist-test-slave-f9xb"
server_key: "36603aae66d3f3bdd7765b5d170109ef"
server_key_iv: "097da602d8cbeb731b1626a0ced2c051"
server_key_version: "kuduclusterkey@0"
I20260926 14:58:08.260138 19597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:08.295037 19597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:08.296918 19597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:08.297043 19597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:08.297251 19597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:08.297565 19597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:58:08.297605 19597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:08.297632 19597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:58:08.297669 19597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:08.304140 19597 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:36887
I20260926 14:58:08.304483 19597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
Sep 26 14:58:08 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434688, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:08.304953 19727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:36887 every 8 connection(s)
I20260926 14:58:08.314599 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 19597
I20260926 14:58:08.314688 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:58:08.314849 7319 external_mini_cluster.cc:1428] Setting key 1c4a10844cf9d997fd5c71773d2b23c5
I20260926 14:58:08.317448 19731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:08.305124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42017 (local address 127.7.37.193:44375)
0926 14:58:08.305285 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:08.305323 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:08.305920 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:08.306312 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:08.306317 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:08.306522 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:08.306774 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:08.306780 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:08.307825 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:08.307827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:08.309052 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:08.309055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:08.309551 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:08.309552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:08.309684 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:08.310187 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:08.310197 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:08.311685 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:08.313586 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:08.313593 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:08.313595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:08.313847 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:08.314059 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:08.314059 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:08.314060 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:08.314173 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:08.314428 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:08.314434 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:08.317013 (+ 2579us) client_negotiation.cc:769] Sending connection context
0926 14:58:08.317101 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 14:58:08.317234 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260926 14:58:08.318130 19730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:08.305149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:44375 (local address 127.7.37.254:42017)
0926 14:58:08.306113 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:08.306115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:08.306123 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:58:08.306147 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:08.306148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:08.306160 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:08.306862 (+ 702us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:08.306901 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:08.307689 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:08.309147 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:08.309455 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:08.311832 (+ 2377us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:08.311882 (+ 50us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:08.311883 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:08.311883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:08.311887 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:08.313498 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:08.313938 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:08.313939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:08.313939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:08.313996 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:08.314238 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:08.314238 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:08.314239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:08.314360 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:08.317904 (+ 3544us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:08.317956 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 14:58:08.318035 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":932,"thread_start_us":73,"threads_started":1}
I20260926 14:58:08.318602 19728 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42017
I20260926 14:58:08.318677 19728 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:08.318831 19728 heartbeater.cc:511] Master 127.7.37.254:42017 requested a full tablet report, sending...
I20260926 14:58:08.319355 19516 ts_manager.cc:194] Registered new tserver with Master: 74c8ab1db5784e6faab3b44de24368a5 (127.7.37.193:36887)
I20260926 14:58:08.320303 19516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:44375
WARNING: no policy specified for kudu/127.7.37.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
I20260926 14:58:08.347453 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.37.212:59435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42017
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:36605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 14:58:08 dist-test-slave-f9xb krb5kdc[19040](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790434688, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 14:58:08.450572 19736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:08.450742 19736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:08.450770 19736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:08.451890 19736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:08.451928 19736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:08.451961 19736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:58:08.453362 19736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:36605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.37.212:59435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:08.453568 19736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:08.453758 19736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:08.458216 19744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:08.458190 19742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:08.458456 19736 server_base.cc:1034] running on GCE node
W20260926 14:58:08.458724 19741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:08.458916 19736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:08.459141 19736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:08.462381 19736 hybrid_clock.cc:648] HybridClock initialized: now 1790434688462352 us; error 48 us; skew 500 ppm
Sep 26 14:58:08 dist-test-slave-f9xb krb5kdc[19040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434688, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:08.465015 19736 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:58:08.465466 19736 webserver.cc:460] Webserver started at http://127.7.37.194:44203/ using document root <none> and password file <none>
I20260926 14:58:08.465708 19736 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:08.465765 19736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:08.465940 19736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 14:58:08 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434688, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
I20260926 14:58:08.480906 19736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "62db0dcbc547494885b652fe360c21f3"
format_stamp: "Formatted at 2026-09-26 14:58:08 on dist-test-slave-f9xb"
server_key: "8e3c5b02fe12ee3458628bdabe41be9c"
server_key_iv: "57759849ed31cd81bfa640882bdf197a"
server_key_version: "kuduclusterkey@0"
I20260926 14:58:08.481293 19736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "62db0dcbc547494885b652fe360c21f3"
format_stamp: "Formatted at 2026-09-26 14:58:08 on dist-test-slave-f9xb"
server_key: "8e3c5b02fe12ee3458628bdabe41be9c"
server_key_iv: "57759849ed31cd81bfa640882bdf197a"
server_key_version: "kuduclusterkey@0"
I20260926 14:58:08.483620 19736 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260926 14:58:08.495159 19752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:08.495337 19736 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:58:08.495414 19736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "62db0dcbc547494885b652fe360c21f3"
format_stamp: "Formatted at 2026-09-26 14:58:08 on dist-test-slave-f9xb"
server_key: "8e3c5b02fe12ee3458628bdabe41be9c"
server_key_iv: "57759849ed31cd81bfa640882bdf197a"
server_key_version: "kuduclusterkey@0"
I20260926 14:58:08.495481 19736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:08.517865 19736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:08.520097 19736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:08.520272 19736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:08.520525 19736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:08.520900 19736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:58:08.520999 19736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:08.521067 19736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:58:08.521121 19736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:08.527307 19736 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:38453
I20260926 14:58:08.527724 19736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:58:08.528366 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 19736
I20260926 14:58:08.528467 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:58:08.528640 7319 external_mini_cluster.cc:1428] Setting key a4167128d438c41e7248a1f0946b94b6
I20260926 14:58:08.540174 19865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:38453 every 8 connection(s)
WARNING: no policy specified for kudu/127.7.37.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.195@KRBTEST.COM" created.
Sep 26 14:58:08 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434688, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:08.553658 19730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:08.541015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:43367 (local address 127.7.37.254:42017)
0926 14:58:08.541069 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:08.541071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:08.542187 (+ 1116us) server_negotiation.cc:407] Connection header received
0926 14:58:08.543024 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:08.543026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:08.543042 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:08.543074 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:08.544002 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:08.544852 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:08.547832 (+ 2980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:08.548157 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:08.548202 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:08.550606 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:08.550607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:08.550608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:08.550615 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:08.552191 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:08.552651 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:08.552652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:08.552653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:08.552709 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:08.552964 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:08.552965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:08.552966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:08.553092 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:08.553135 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:08.553486 (+ 351us) server_negotiation.cc:299] Negotiation successful
0926 14:58:08.553568 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 14:58:08.554296 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:08.540767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42017 (local address 127.7.37.194:43367)
0926 14:58:08.542061 (+ 1294us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:08.542105 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:08.542774 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:08.543258 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:08.543262 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:08.543492 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:08.543889 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:08.543896 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:08.546470 (+ 2574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:08.546471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:08.547718 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:08.547720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:08.548298 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:08.548298 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:08.548448 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:08.548956 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:08.548966 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:08.550507 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:08.552281 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:08.552287 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:08.552289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:08.552566 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:08.552772 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:08.552773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:08.552773 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:08.552892 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:08.553180 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:08.553185 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:08.553332 (+ 147us) client_negotiation.cc:769] Sending connection context
0926 14:58:08.553947 (+ 615us) client_negotiation.cc:240] Negotiation successful
0926 14:58:08.554088 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1251,"thread_start_us":73,"threads_started":1}
I20260926 14:58:08.554891 19866 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42017
I20260926 14:58:08.554972 19866 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:08.555133 19866 heartbeater.cc:511] Master 127.7.37.254:42017 requested a full tablet report, sending...
I20260926 14:58:08.555454 19516 ts_manager.cc:194] Registered new tserver with Master: 62db0dcbc547494885b652fe360c21f3 (127.7.37.194:38453)
I20260926 14:58:08.555941 19516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:43367
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
I20260926 14:58:08.570289 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.37.212:59435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42017
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:36605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:58:08.677166 19871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:08.677343 19871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:08.677372 19871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:08.678493 19871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:08.678534 19871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:08.678566 19871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:58:08.680212 19871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:36605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.37.212:59435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:08.680433 19871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:08.680657 19871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:08.685401 19877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:08.685406 19879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:08.685468 19876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:08.685804 19871 server_base.cc:1034] running on GCE node
I20260926 14:58:08.685981 19871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:08.686209 19871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:08.687368 19871 hybrid_clock.cc:648] HybridClock initialized: now 1790434688687348 us; error 30 us; skew 500 ppm
Sep 26 14:58:08 dist-test-slave-f9xb krb5kdc[19040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434688, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:08.690174 19871 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:58:08.690531 19871 webserver.cc:460] Webserver started at http://127.7.37.195:38887/ using document root <none> and password file <none>
I20260926 14:58:08.690713 19871 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:08.690757 19871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:08.690871 19871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 14:58:08 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434688, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
I20260926 14:58:08.704316 19871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "930c76e3245e44e2913494cca530ccfa"
format_stamp: "Formatted at 2026-09-26 14:58:08 on dist-test-slave-f9xb"
server_key: "b423b6347ed545602f296b5fc9cbf785"
server_key_iv: "5f61b2700215964f25d295cd32b43256"
server_key_version: "kuduclusterkey@0"
I20260926 14:58:08.704896 19871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "930c76e3245e44e2913494cca530ccfa"
format_stamp: "Formatted at 2026-09-26 14:58:08 on dist-test-slave-f9xb"
server_key: "b423b6347ed545602f296b5fc9cbf785"
server_key_iv: "5f61b2700215964f25d295cd32b43256"
server_key_version: "kuduclusterkey@0"
I20260926 14:58:08.707455 19871 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260926 14:58:08.739567 19886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:08.739732 19871 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.001s sys 0.000s
I20260926 14:58:08.739845 19871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "930c76e3245e44e2913494cca530ccfa"
format_stamp: "Formatted at 2026-09-26 14:58:08 on dist-test-slave-f9xb"
server_key: "b423b6347ed545602f296b5fc9cbf785"
server_key_iv: "5f61b2700215964f25d295cd32b43256"
server_key_version: "kuduclusterkey@0"
I20260926 14:58:08.739939 19871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:08.758446 19871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:08.760308 19871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:08.760501 19871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:08.760727 19871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:08.761101 19871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:58:08.761193 19871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:08.761264 19871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:58:08.761296 19871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:08.767472 19871 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:32847
I20260926 14:58:08.767832 19871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:58:08.767977 19999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:32847 every 8 connection(s)
I20260926 14:58:08.775547 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 19871
I20260926 14:58:08.775643 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:58:08.776252 7319 external_mini_cluster.cc:1428] Setting key 9e099c1e54ff6f4a05034175e3e1ddaf
Sep 26 14:58:08 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434688, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:08.780258 19730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:08.772238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:35495 (local address 127.7.37.254:42017)
0926 14:58:08.772282 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:08.772284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:08.772569 (+ 285us) server_negotiation.cc:407] Connection header received
0926 14:58:08.773040 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:08.773042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:08.773083 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:08.773131 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:08.773628 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:08.774117 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:08.775094 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:08.775291 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:08.775365 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:08.777568 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:08.777569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:08.777570 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:08.777576 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:08.779025 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:08.779355 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:08.779357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:08.779357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:08.779411 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:08.779618 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:08.779619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:08.779619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:08.779734 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:08.779821 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:08.780099 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 14:58:08.780171 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 14:58:08.780448 20002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:08.772242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42017 (local address 127.7.37.195:35495)
0926 14:58:08.772458 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:08.772490 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:08.772950 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:08.773201 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:08.773204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:08.773362 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:08.773547 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:08.773551 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:08.774191 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:08.774192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:08.775015 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:08.775018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:08.775384 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:08.775385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:08.775525 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:08.776055 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:08.776064 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:08.777491 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:08.779095 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:08.779101 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:08.779103 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:08.779295 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:08.779467 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:08.779468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:08.779468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:08.779567 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:08.779848 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:08.779854 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:08.779979 (+ 125us) client_negotiation.cc:769] Sending connection context
0926 14:58:08.780066 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 14:58:08.780190 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":73,"threads_started":1}
I20260926 14:58:08.780864 20000 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42017
I20260926 14:58:08.780946 20000 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:08.781100 20000 heartbeater.cc:511] Master 127.7.37.254:42017 requested a full tablet report, sending...
I20260926 14:58:08.781424 19516 ts_manager.cc:194] Registered new tserver with Master: 930c76e3245e44e2913494cca530ccfa (127.7.37.195:32847)
I20260926 14:58:08.781765 19516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:35495
I20260926 14:58:08.787762 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 14:58:08.788514 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 19597
I20260926 14:58:08.793560 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 19736
I20260926 14:58:08.798324 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 19871
I20260926 14:58:08.803412 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 19484
I20260926 14:58:08.807370 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:42017
--webserver_interface=127.7.37.254
--webserver_port=36817
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:36605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:42017
--ranger_config_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.37.212:59435/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.37.212:59435/kms with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:58:08.881999 20003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:08.882169 20003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:08.882200 20003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:08.883278 20003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:58:08.883315 20003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:08.883333 20003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:08.883348 20003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:58:08.883363 20003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:58:08.884824 20003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:36605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.37.212:59435/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:42017
--ranger_config_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:42017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=36817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:08.885021 20003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:08.885195 20003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:08.889329 20008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:08.889405 20011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:08.889451 20009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:08.889581 20003 server_base.cc:1034] running on GCE node
I20260926 14:58:08.889784 20003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:08.890053 20003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:08.891191 20003 hybrid_clock.cc:648] HybridClock initialized: now 1790434688891180 us; error 28 us; skew 500 ppm
Sep 26 14:58:08 dist-test-slave-f9xb krb5kdc[19040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434688, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:08.893548 20003 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:58:08.893953 20003 webserver.cc:460] Webserver started at http://127.7.37.254:36817/ using document root <none> and password file <none>
I20260926 14:58:08.894148 20003 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:08.894194 20003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 14:58:08 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434688, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
I20260926 14:58:08.959470 20003 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:58:08.960270 20019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:08.960436 20003 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 14:58:08.960523 20003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "55ccc7022b4d46d99ff94743d6e6929b"
format_stamp: "Formatted at 2026-09-26 14:58:07 on dist-test-slave-f9xb"
server_key: "8f65cfa3e940e490345e32147a5309aa"
server_key_iv: "2e9fa392bb85effac86a3ab2aa406c1f"
server_key_version: "kuduclusterkey@0"
I20260926 14:58:08.960866 20003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:08.979091 20003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:08.980840 20003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:08.980962 20003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:08.984777 20003 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:42017
I20260926 14:58:08.984822 20071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:42017 every 8 connection(s)
I20260926 14:58:08.985140 20003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:58:08.986235 20072 sys_catalog.cc:263] Verifying existing consensus state
I20260926 14:58:08.986822 20072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b: Bootstrap starting.
I20260926 14:58:08.992065 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 20003
I20260926 14:58:08.992343 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.37.212:59435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:36887
--local_ip_for_outbound_sockets=127.7.37.193
--tserver_master_addrs=127.7.37.254:42017
--webserver_port=36537
--webserver_interface=127.7.37.193
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:36605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.37.212:59435/kms with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 14:58:08.994757 20072 log.cc:824] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:08.996402 20072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 14:58:08.996639 20072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b: Bootstrap complete.
I20260926 14:58:08.998696 20072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ccc7022b4d46d99ff94743d6e6929b" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42017 } }
I20260926 14:58:08.998910 20072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55ccc7022b4d46d99ff94743d6e6929b, State: Initialized, Role: FOLLOWER
I20260926 14:58:08.999040 20072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ccc7022b4d46d99ff94743d6e6929b" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42017 } }
I20260926 14:58:08.999100 20072 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:58:08.999130 20072 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:58:08.999172 20072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [term 1 FOLLOWER]: Advancing to term 2
I20260926 14:58:09.000135 20072 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ccc7022b4d46d99ff94743d6e6929b" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42017 } }
I20260926 14:58:09.000216 20072 leader_election.cc:302] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55ccc7022b4d46d99ff94743d6e6929b; no voters:
I20260926 14:58:09.000368 20072 leader_election.cc:288] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 14:58:09.000409 20076 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [term 2 FOLLOWER]: Leader election won for term 2
I20260926 14:58:09.000528 20076 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [term 2 LEADER]: Becoming Leader. State: Replica: 55ccc7022b4d46d99ff94743d6e6929b, State: Running, Role: LEADER
I20260926 14:58:09.000614 20072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:58:09.000614 20076 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ccc7022b4d46d99ff94743d6e6929b" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42017 } }
I20260926 14:58:09.000921 20078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "55ccc7022b4d46d99ff94743d6e6929b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ccc7022b4d46d99ff94743d6e6929b" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42017 } } }
I20260926 14:58:09.001008 20078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [sys.catalog]: This master's current role is: LEADER
I20260926 14:58:09.000916 20077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55ccc7022b4d46d99ff94743d6e6929b. Latest consensus state: current_term: 2 leader_uuid: "55ccc7022b4d46d99ff94743d6e6929b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ccc7022b4d46d99ff94743d6e6929b" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42017 } } }
I20260926 14:58:09.001174 20077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b [sys.catalog]: This master's current role is: LEADER
I20260926 14:58:09.001266 20081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:58:09.001722 20081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:58:09.002034 20072 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 14:58:09.002305 20081 catalog_manager.cc:1259] Loaded cluster ID: d1aba86e54184be982f2185e0c95239c
I20260926 14:58:09.002347 20081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:58:09.005177 20081 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:58:09.005649 20081 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b: Loaded TSK: 0
I20260926 14:58:09.005883 20081 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 14:58:09.101609 20074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:09.102022 20074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:09.102101 20074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:09.103788 20074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:09.103899 20074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:09.103979 20074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:58:09.106099 20074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:36605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.37.212:59435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:36887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=36537
--tserver_master_addrs=127.7.37.254:42017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:09.106318 20074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:09.106570 20074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:09.112526 20109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:09.112519 20112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:09.112722 20110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:09.112972 20074 server_base.cc:1034] running on GCE node
I20260926 14:58:09.113173 20074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:09.113425 20074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:09.114593 20074 hybrid_clock.cc:648] HybridClock initialized: now 1790434689114565 us; error 42 us; skew 500 ppm
Sep 26 14:58:09 dist-test-slave-f9xb krb5kdc[19040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434689, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:09.117653 20074 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:58:09.118139 20074 webserver.cc:460] Webserver started at http://127.7.37.193:36537/ using document root <none> and password file <none>
I20260926 14:58:09.118358 20074 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:09.118420 20074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 14:58:09 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434689, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
I20260926 14:58:09.142168 20074 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:58:09.143150 20120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:09.143293 20074 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 14:58:09.143364 20074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "74c8ab1db5784e6faab3b44de24368a5"
format_stamp: "Formatted at 2026-09-26 14:58:08 on dist-test-slave-f9xb"
server_key: "36603aae66d3f3bdd7765b5d170109ef"
server_key_iv: "097da602d8cbeb731b1626a0ced2c051"
server_key_version: "kuduclusterkey@0"
I20260926 14:58:09.143787 20074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:09.159834 20074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:09.161590 20074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:09.161696 20074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:09.161880 20074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:09.162168 20074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:58:09.162199 20074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:09.162218 20074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:58:09.162240 20074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:09.168555 20074 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:36887
I20260926 14:58:09.168634 20233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:36887 every 8 connection(s)
I20260926 14:58:09.168963 20074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:58:09.172823 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 20074
I20260926 14:58:09.173091 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.37.212:59435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:38453
--local_ip_for_outbound_sockets=127.7.37.194
--tserver_master_addrs=127.7.37.254:42017
--webserver_port=44203
--webserver_interface=127.7.37.194
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:36605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.37.212:59435/kms with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 14:58:09 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434689, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:09.183969 20236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:09.169634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42017 (local address 127.7.37.193:41749)
0926 14:58:09.169778 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:09.169824 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:09.170301 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:09.170862 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:09.170867 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:09.171079 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:09.171336 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:09.171342 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:09.172848 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:09.172849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:09.173966 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:09.173969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:09.174981 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:09.174982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:09.175114 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:09.175587 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:09.175596 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:09.177069 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:09.182083 (+ 5014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:09.182089 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:09.182091 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:09.182368 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:09.182651 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:09.182652 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:09.182653 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:09.182779 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:09.183274 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:09.183281 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:09.183438 (+ 157us) client_negotiation.cc:769] Sending connection context
0926 14:58:09.183556 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 14:58:09.183704 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
I20260926 14:58:09.184445 20237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:09.169715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:41749 (local address 127.7.37.254:42017)
0926 14:58:09.170460 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:09.170467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:09.170491 (+ 24us) server_negotiation.cc:407] Connection header received
0926 14:58:09.170595 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:09.170599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:09.170647 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:09.170768 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:09.171457 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:09.172729 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:09.174272 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:09.174859 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:09.175358 (+ 499us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:09.177180 (+ 1822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:09.177184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:09.177184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:09.177194 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:09.181984 (+ 4790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:09.182465 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:09.182467 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:09.182468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:09.182572 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:09.182867 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:09.182868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:09.182868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:09.183174 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:09.183284 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:09.184144 (+ 860us) server_negotiation.cc:299] Negotiation successful
0926 14:58:09.184260 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":75,"threads_started":1}
I20260926 14:58:09.185027 20234 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42017
I20260926 14:58:09.185118 20234 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:09.185302 20234 heartbeater.cc:511] Master 127.7.37.254:42017 requested a full tablet report, sending...
W20260926 14:58:09.186105 20234 heartbeater.cc:644] Failed to heartbeat to 127.7.37.254:42017 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 14:58:09.186174 20234 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:09.186232 20234 heartbeater.cc:511] Master 127.7.37.254:42017 requested a full tablet report, sending...
I20260926 14:58:09.186518 20234 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:09.186587 20234 heartbeater.cc:511] Master 127.7.37.254:42017 requested a full tablet report, sending...
W20260926 14:58:09.186842 20234 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 14:58:09.276453 20238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:09.276629 20238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:09.276659 20238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:09.277761 20238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:09.277799 20238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:09.277833 20238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:58:09.279243 20238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:36605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.37.212:59435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:38453
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=44203
--tserver_master_addrs=127.7.37.254:42017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:09.279446 20238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:09.279654 20238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:09.284297 20244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:09.284322 20246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:09.285221 20243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:09.285393 20238 server_base.cc:1034] running on GCE node
I20260926 14:58:09.285589 20238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:09.285794 20238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:09.286988 20238 hybrid_clock.cc:648] HybridClock initialized: now 1790434689286976 us; error 26 us; skew 500 ppm
Sep 26 14:58:09 dist-test-slave-f9xb krb5kdc[19040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434689, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:09.289655 20238 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:58:09.290009 20238 webserver.cc:460] Webserver started at http://127.7.37.194:44203/ using document root <none> and password file <none>
I20260926 14:58:09.290200 20238 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:09.290257 20238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 14:58:09 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434689, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
I20260926 14:58:09.306461 20238 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260926 14:58:09.307399 20254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:09.307615 20238 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 14:58:09.307703 20238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "62db0dcbc547494885b652fe360c21f3"
format_stamp: "Formatted at 2026-09-26 14:58:08 on dist-test-slave-f9xb"
server_key: "8e3c5b02fe12ee3458628bdabe41be9c"
server_key_iv: "57759849ed31cd81bfa640882bdf197a"
server_key_version: "kuduclusterkey@0"
I20260926 14:58:09.308072 20238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:09.327207 20238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:09.329210 20238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:09.329357 20238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:09.329582 20238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:09.329949 20238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:58:09.329996 20238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:09.330041 20238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:58:09.330065 20238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:09.336472 20238 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:38453
I20260926 14:58:09.336539 20367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:38453 every 8 connection(s)
I20260926 14:58:09.336858 20238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:58:09.339941 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 20238
I20260926 14:58:09.340196 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.37.212:59435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:32847
--local_ip_for_outbound_sockets=127.7.37.195
--tserver_master_addrs=127.7.37.254:42017
--webserver_port=38887
--webserver_interface=127.7.37.195
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:36605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.37.212:59435/kms with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 14:58:09 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434689, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:09.349731 20237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:09.337695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:41537 (local address 127.7.37.254:42017)
0926 14:58:09.337739 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:09.337741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:09.337961 (+ 220us) server_negotiation.cc:407] Connection header received
0926 14:58:09.338425 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:09.338426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:09.338440 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:09.338495 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:09.338998 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:09.339789 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:09.341223 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:09.341548 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:09.341597 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:09.347120 (+ 5523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:09.347121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:09.347122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:09.347129 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:09.348540 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:09.348886 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:09.348887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:09.348888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:09.348943 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:09.349149 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:09.349149 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:09.349150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:09.349257 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:09.349310 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:09.349548 (+ 238us) server_negotiation.cc:299] Negotiation successful
0926 14:58:09.349641 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 14:58:09.349912 20370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:09.337701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42017 (local address 127.7.37.194:41537)
0926 14:58:09.337843 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:09.337888 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:09.338328 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:09.338562 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:09.338566 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:09.338714 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:09.338891 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:09.338895 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:09.339897 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:09.339898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:09.341117 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:09.341120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:09.341700 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:09.341701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:09.341833 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:09.342326 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:09.342335 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:09.347009 (+ 4674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:09.348615 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:09.348621 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:09.348623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:09.348822 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:09.348997 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:09.348997 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:09.348998 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:09.349096 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:09.349309 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:09.349315 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:09.349439 (+ 124us) client_negotiation.cc:769] Sending connection context
0926 14:58:09.349521 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 14:58:09.349654 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
I20260926 14:58:09.350366 20368 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42017
I20260926 14:58:09.350453 20368 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:09.350608 20368 heartbeater.cc:511] Master 127.7.37.254:42017 requested a full tablet report, sending...
W20260926 14:58:09.351152 20368 heartbeater.cc:644] Failed to heartbeat to 127.7.37.254:42017 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 14:58:09.351208 20368 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:09.351267 20368 heartbeater.cc:511] Master 127.7.37.254:42017 requested a full tablet report, sending...
I20260926 14:58:09.351555 20368 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:09.351624 20368 heartbeater.cc:511] Master 127.7.37.254:42017 requested a full tablet report, sending...
W20260926 14:58:09.351900 20368 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 14:58:09.455785 20371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:09.456080 20371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:09.456139 20371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:09.457795 20371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:09.457883 20371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:09.457947 20371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:58:09.460007 20371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:36605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.37.212:59435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:32847
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=38887
--tserver_master_addrs=127.7.37.254:42017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:09.460361 20371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:09.460625 20371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:09.466835 20378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:09.466928 20377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:09.467583 20380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:09.467911 20371 server_base.cc:1034] running on GCE node
I20260926 14:58:09.468097 20371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:09.468343 20371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:09.469496 20371 hybrid_clock.cc:648] HybridClock initialized: now 1790434689469475 us; error 34 us; skew 500 ppm
Sep 26 14:58:09 dist-test-slave-f9xb krb5kdc[19040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434689, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:09.472714 20371 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:58:09.473199 20371 webserver.cc:460] Webserver started at http://127.7.37.195:38887/ using document root <none> and password file <none>
I20260926 14:58:09.473490 20371 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:09.473553 20371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 14:58:09 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434689, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
I20260926 14:58:09.487064 20371 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:58:09.489403 20388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:09.489598 20371 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260926 14:58:09.489719 20371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "930c76e3245e44e2913494cca530ccfa"
format_stamp: "Formatted at 2026-09-26 14:58:08 on dist-test-slave-f9xb"
server_key: "b423b6347ed545602f296b5fc9cbf785"
server_key_iv: "5f61b2700215964f25d295cd32b43256"
server_key_version: "kuduclusterkey@0"
I20260926 14:58:09.491575 20371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:09.509872 20371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:09.511811 20371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:09.511955 20371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:09.512182 20371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:09.512585 20371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:58:09.512637 20371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:09.512672 20371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:58:09.512697 20371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:09.518935 20371 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:32847
I20260926 14:58:09.519371 20371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:58:09.520283 20501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:32847 every 8 connection(s)
Sep 26 14:58:09 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434689, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:09.527479 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 20371
I20260926 14:58:09.529588 20237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:09.520380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:51595 (local address 127.7.37.254:42017)
0926 14:58:09.520448 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:09.520450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:09.520632 (+ 182us) server_negotiation.cc:407] Connection header received
0926 14:58:09.521288 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:09.521289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:09.521302 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:09.521333 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:09.522101 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:09.522865 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:09.524328 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:09.524632 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:09.524671 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:09.526898 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:09.526899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:09.526900 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:09.526907 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:09.528347 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:09.528754 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:09.528755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:09.528756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:09.528814 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:09.529034 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:09.529035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:09.529035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:09.529140 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:09.529193 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:09.529431 (+ 238us) server_negotiation.cc:299] Negotiation successful
0926 14:58:09.529503 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 14:58:09.529740 20504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:09.520123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42017 (local address 127.7.37.195:51595)
0926 14:58:09.520520 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:09.520561 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:09.521162 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:09.521509 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:09.521513 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:09.521714 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:09.521991 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:09.521997 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:09.522979 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:09.522980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:09.524231 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:09.524234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:09.524766 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:09.524767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:09.524897 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:09.525395 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:09.525404 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:09.526818 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:09.528419 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:09.528425 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:09.528427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:09.528694 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:09.528878 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:09.528879 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:09.528879 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:09.528980 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:09.529192 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:09.529197 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:09.529321 (+ 124us) client_negotiation.cc:769] Sending connection context
0926 14:58:09.529405 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 14:58:09.529539 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":74,"threads_started":1}
I20260926 14:58:09.530117 20502 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42017
I20260926 14:58:09.530200 20502 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:09.530351 20502 heartbeater.cc:511] Master 127.7.37.254:42017 requested a full tablet report, sending...
W20260926 14:58:09.530949 20502 heartbeater.cc:644] Failed to heartbeat to 127.7.37.254:42017 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 14:58:09.531008 20502 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:09.531070 20502 heartbeater.cc:511] Master 127.7.37.254:42017 requested a full tablet report, sending...
I20260926 14:58:09.531380 20502 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:09.531446 20502 heartbeater.cc:511] Master 127.7.37.254:42017 requested a full tablet report, sending...
W20260926 14:58:09.531776 20502 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 14:58:09.540760 20506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:09.528216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35430 (local address 127.7.37.254:42017)
0926 14:58:09.528306 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:09.528307 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:09.528753 (+ 446us) server_negotiation.cc:407] Connection header received
0926 14:58:09.528771 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:09.528772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:09.528784 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:09.528849 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:09.529796 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:09.531323 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:09.532890 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:09.533082 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:09.533128 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:09.534594 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:09.534596 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:09.534596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:09.534602 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:09.538010 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:09.538384 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:09.538385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:09.538386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:09.538446 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:09.539819 (+ 1373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:09.539820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:09.539821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:09.539953 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:09.540126 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:09.540585 (+ 459us) server_negotiation.cc:299] Negotiation successful
0926 14:58:09.540667 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":47,"threads_started":1}
Sep 26 14:58:10 dist-test-slave-f9xb krb5kdc[19040](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790434690, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:10.187012 20234 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:10.187148 20234 heartbeater.cc:511] Master 127.7.37.254:42017 requested a full tablet report, sending...
I20260926 14:58:10.187491 20036 ts_manager.cc:194] Registered new tserver with Master: 74c8ab1db5784e6faab3b44de24368a5 (127.7.37.193:36887)
I20260926 14:58:10.188521 20036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:41749
I20260926 14:58:10.352051 20368 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:10.352186 20368 heartbeater.cc:511] Master 127.7.37.254:42017 requested a full tablet report, sending...
I20260926 14:58:10.352445 20036 ts_manager.cc:194] Registered new tserver with Master: 62db0dcbc547494885b652fe360c21f3 (127.7.37.194:38453)
I20260926 14:58:10.352977 20036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:41537
Sep 26 14:58:10 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434690, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
I20260926 14:58:10.531937 20502 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:10.532065 20502 heartbeater.cc:511] Master 127.7.37.254:42017 requested a full tablet report, sending...
I20260926 14:58:10.532315 20036 ts_manager.cc:194] Registered new tserver with Master: 930c76e3245e44e2913494cca530ccfa (127.7.37.195:32847)
I20260926 14:58:10.532828 20036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:51595
I20260926 14:58:10.538918 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 14:58:10.549974 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:10.542058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35436 (local address 127.7.37.254:42017)
0926 14:58:10.542177 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:10.542180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:10.542768 (+ 588us) server_negotiation.cc:407] Connection header received
0926 14:58:10.542871 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:10.542872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:10.542884 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:10.542922 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:10.543597 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.544347 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.545300 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.545610 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.545688 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:10.547170 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:10.547172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:10.547172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:10.547179 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:10.548809 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:10.549245 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:10.549246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:10.549246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:10.549309 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:10.549481 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:10.549481 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:10.549482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:10.549601 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:10.549692 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:10.549789 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 14:58:10.549868 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
I20260926 14:58:10.552436 20036 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:58:10.554656 20036 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:58:10.562868 20529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:10.557518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:32847 (local address 127.0.0.1:36110)
0926 14:58:10.557866 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:10.557906 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:10.557974 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:10.559057 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:10.559058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:10.559060 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:10.559121 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:10.559122 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.560319 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.560320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:10.561622 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:10.561625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.562566 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.562567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:10.562648 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:10.562660 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 14:58:10.562744 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 14:58:10.562789 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":71,"threads_started":1}
I20260926 14:58:10.563217 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:10.557755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36110 (local address 127.7.37.195:32847)
0926 14:58:10.558856 (+ 1101us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:10.558861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:10.558869 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:58:10.558900 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:10.558902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:10.558917 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:10.558985 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:10.559199 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.560225 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.561719 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.562419 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.562481 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:10.562534 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:10.563105 (+ 571us) server_negotiation.cc:299] Negotiation successful
0926 14:58:10.563147 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1057,"thread_start_us":59,"threads_started":1}
I20260926 14:58:10.564087 20418 tablet_service.cc:1467] Processing CreateTablet for tablet 49182666b4bd4d32a29f73f32fb1e505 (DEFAULT_TABLE table=test-table [id=8c2f30f76ebc47d08f5bd8b42c01698b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:58:10.564417 20418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49182666b4bd4d32a29f73f32fb1e505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:10.564778 20532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:10.558112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:38453 (local address 127.0.0.1:56086)
0926 14:58:10.558683 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:10.558692 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:10.558744 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:10.559147 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:10.559148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:10.559150 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:10.559197 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:10.559199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.560769 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.560770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:10.561836 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:10.561839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.564523 (+ 2684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.564524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:10.564622 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:10.564626 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:58:10.564672 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 14:58:10.564711 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":52,"threads_started":1}
I20260926 14:58:10.565035 20533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:10.558208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56086 (local address 127.7.37.194:38453)
0926 14:58:10.558352 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:10.558356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:10.558746 (+ 390us) server_negotiation.cc:407] Connection header received
0926 14:58:10.558971 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:10.558974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:10.558991 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:10.559062 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:10.559273 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.560686 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.561949 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.563259 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.563323 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:10.563375 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:10.564905 (+ 1530us) server_negotiation.cc:299] Negotiation successful
0926 14:58:10.564959 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":53,"threads_started":1}
I20260926 14:58:10.565615 20295 tablet_service.cc:1467] Processing CreateTablet for tablet 49182666b4bd4d32a29f73f32fb1e505 (DEFAULT_TABLE table=test-table [id=8c2f30f76ebc47d08f5bd8b42c01698b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:58:10.565932 20295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49182666b4bd4d32a29f73f32fb1e505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:10.567597 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:10.557965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:36887 (local address 127.0.0.1:59340)
0926 14:58:10.558098 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:10.558107 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:10.558172 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:10.558826 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:10.558829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:10.558832 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:10.558889 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:10.558890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.565169 (+ 6279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.565170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:10.566243 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:10.566246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.567397 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.567398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:10.567470 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:10.567474 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:58:10.567504 (+ 30us) client_negotiation.cc:240] Negotiation successful
0926 14:58:10.567537 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260926 14:58:10.567968 20534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:10.558361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59340 (local address 127.7.37.193:36887)
0926 14:58:10.558521 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:10.558525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:10.558536 (+ 11us) server_negotiation.cc:407] Connection header received
0926 14:58:10.558623 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:10.558626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:10.558641 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:10.562793 (+ 4152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:10.562863 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.564441 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.566567 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.567302 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.567760 (+ 458us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:10.567823 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:10.567848 (+ 25us) server_negotiation.cc:299] Negotiation successful
0926 14:58:10.567884 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260926 14:58:10.568529 20168 tablet_service.cc:1467] Processing CreateTablet for tablet 49182666b4bd4d32a29f73f32fb1e505 (DEFAULT_TABLE table=test-table [id=8c2f30f76ebc47d08f5bd8b42c01698b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:58:10.568789 20168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49182666b4bd4d32a29f73f32fb1e505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:10.568944 20536 tablet_bootstrap.cc:492] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3: Bootstrap starting.
I20260926 14:58:10.569564 20536 tablet_bootstrap.cc:654] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:10.569900 20536 log.cc:824] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:10.570634 20536 tablet_bootstrap.cc:492] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3: No bootstrap required, opened a new log
I20260926 14:58:10.570698 20536 ts_tablet_manager.cc:1397] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:58:10.571414 20538 tablet_bootstrap.cc:492] T 49182666b4bd4d32a29f73f32fb1e505 P 74c8ab1db5784e6faab3b44de24368a5: Bootstrap starting.
I20260926 14:58:10.572088 20538 tablet_bootstrap.cc:654] T 49182666b4bd4d32a29f73f32fb1e505 P 74c8ab1db5784e6faab3b44de24368a5: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:10.572428 20538 log.cc:824] T 49182666b4bd4d32a29f73f32fb1e505 P 74c8ab1db5784e6faab3b44de24368a5: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:10.572710 20536 raft_consensus.cc:348] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "930c76e3245e44e2913494cca530ccfa" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 32847 } } peers { permanent_uuid: "74c8ab1db5784e6faab3b44de24368a5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 36887 } } peers { permanent_uuid: "62db0dcbc547494885b652fe360c21f3" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38453 } }
I20260926 14:58:10.572876 20536 raft_consensus.cc:374] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:10.572906 20536 raft_consensus.cc:729] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62db0dcbc547494885b652fe360c21f3, State: Initialized, Role: FOLLOWER
I20260926 14:58:10.573037 20536 consensus_queue.cc:260] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "930c76e3245e44e2913494cca530ccfa" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 32847 } } peers { permanent_uuid: "74c8ab1db5784e6faab3b44de24368a5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 36887 } } peers { permanent_uuid: "62db0dcbc547494885b652fe360c21f3" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38453 } }
I20260926 14:58:10.573267 20536 ts_tablet_manager.cc:1428] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 14:58:10.573374 20368 heartbeater.cc:503] Master 127.7.37.254:42017 was elected leader, sending a full tablet report...
I20260926 14:58:10.574361 20538 tablet_bootstrap.cc:492] T 49182666b4bd4d32a29f73f32fb1e505 P 74c8ab1db5784e6faab3b44de24368a5: No bootstrap required, opened a new log
I20260926 14:58:10.574463 20538 ts_tablet_manager.cc:1397] T 49182666b4bd4d32a29f73f32fb1e505 P 74c8ab1db5784e6faab3b44de24368a5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260926 14:58:10.576748 20538 raft_consensus.cc:348] T 49182666b4bd4d32a29f73f32fb1e505 P 74c8ab1db5784e6faab3b44de24368a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "930c76e3245e44e2913494cca530ccfa" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 32847 } } peers { permanent_uuid: "74c8ab1db5784e6faab3b44de24368a5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 36887 } } peers { permanent_uuid: "62db0dcbc547494885b652fe360c21f3" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38453 } }
I20260926 14:58:10.576961 20538 raft_consensus.cc:374] T 49182666b4bd4d32a29f73f32fb1e505 P 74c8ab1db5784e6faab3b44de24368a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:10.577031 20538 raft_consensus.cc:729] T 49182666b4bd4d32a29f73f32fb1e505 P 74c8ab1db5784e6faab3b44de24368a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74c8ab1db5784e6faab3b44de24368a5, State: Initialized, Role: FOLLOWER
I20260926 14:58:10.577184 20538 consensus_queue.cc:260] T 49182666b4bd4d32a29f73f32fb1e505 P 74c8ab1db5784e6faab3b44de24368a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "930c76e3245e44e2913494cca530ccfa" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 32847 } } peers { permanent_uuid: "74c8ab1db5784e6faab3b44de24368a5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 36887 } } peers { permanent_uuid: "62db0dcbc547494885b652fe360c21f3" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38453 } }
I20260926 14:58:10.577440 20538 ts_tablet_manager.cc:1428] T 49182666b4bd4d32a29f73f32fb1e505 P 74c8ab1db5784e6faab3b44de24368a5: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260926 14:58:10.577564 20234 heartbeater.cc:503] Master 127.7.37.254:42017 was elected leader, sending a full tablet report...
I20260926 14:58:10.579856 20535 tablet_bootstrap.cc:492] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa: Bootstrap starting.
I20260926 14:58:10.580302 20535 tablet_bootstrap.cc:654] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:10.580542 20535 log.cc:824] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:10.581148 20535 tablet_bootstrap.cc:492] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa: No bootstrap required, opened a new log
I20260926 14:58:10.581202 20535 ts_tablet_manager.cc:1397] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:58:10.582628 20535 raft_consensus.cc:348] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "930c76e3245e44e2913494cca530ccfa" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 32847 } } peers { permanent_uuid: "74c8ab1db5784e6faab3b44de24368a5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 36887 } } peers { permanent_uuid: "62db0dcbc547494885b652fe360c21f3" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38453 } }
I20260926 14:58:10.582728 20535 raft_consensus.cc:374] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:10.582746 20535 raft_consensus.cc:729] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 930c76e3245e44e2913494cca530ccfa, State: Initialized, Role: FOLLOWER
I20260926 14:58:10.582820 20535 consensus_queue.cc:260] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "930c76e3245e44e2913494cca530ccfa" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 32847 } } peers { permanent_uuid: "74c8ab1db5784e6faab3b44de24368a5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 36887 } } peers { permanent_uuid: "62db0dcbc547494885b652fe360c21f3" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38453 } }
I20260926 14:58:10.583022 20535 ts_tablet_manager.cc:1428] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:58:10.583051 20502 heartbeater.cc:503] Master 127.7.37.254:42017 was elected leader, sending a full tablet report...
W20260926 14:58:10.587867 20369 tablet.cc:2369] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 14:58:10 dist-test-slave-f9xb krb5kdc[19040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434690, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for HTTP/127.7.37.212@KRBTEST.COM
W20260926 14:58:10.670291 20235 tablet.cc:2369] T 49182666b4bd4d32a29f73f32fb1e505 P 74c8ab1db5784e6faab3b44de24368a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:58:10.724105 20540 raft_consensus.cc:482] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:58:10.724238 20540 raft_consensus.cc:504] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "930c76e3245e44e2913494cca530ccfa" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 32847 } } peers { permanent_uuid: "74c8ab1db5784e6faab3b44de24368a5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 36887 } } peers { permanent_uuid: "62db0dcbc547494885b652fe360c21f3" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38453 } }
I20260926 14:58:10.724623 20540 leader_election.cc:288] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 930c76e3245e44e2913494cca530ccfa (127.7.37.195:32847), 74c8ab1db5784e6faab3b44de24368a5 (127.7.37.193:36887)
I20260926 14:58:10.729158 20534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:10.725203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:34253 (local address 127.7.37.193:36887)
0926 14:58:10.725247 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:10.725249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:10.725702 (+ 453us) server_negotiation.cc:407] Connection header received
0926 14:58:10.725807 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:10.725808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:10.725820 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:10.725848 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:10.726085 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.726830 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.728234 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.728769 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.728812 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:10.728821 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:10.729034 (+ 213us) server_negotiation.cc:299] Negotiation successful
0926 14:58:10.729089 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:58:10.729566 20546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:10.724989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:36887 (local address 127.7.37.194:34253)
0926 14:58:10.725637 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:10.725648 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:10.725733 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:10.725916 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:10.725917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:10.725918 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:10.726007 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:10.726009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.726926 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.726938 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:10.728145 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:10.728148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.728884 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.728885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:10.728961 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:10.728965 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:58:10.729438 (+ 473us) client_negotiation.cc:240] Negotiation successful
0926 14:58:10.729475 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":89,"threads_started":1}
I20260926 14:58:10.730038 20188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49182666b4bd4d32a29f73f32fb1e505" candidate_uuid: "62db0dcbc547494885b652fe360c21f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74c8ab1db5784e6faab3b44de24368a5" is_pre_election: true
I20260926 14:58:10.730199 20188 raft_consensus.cc:2454] T 49182666b4bd4d32a29f73f32fb1e505 P 74c8ab1db5784e6faab3b44de24368a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62db0dcbc547494885b652fe360c21f3 in term 0.
I20260926 14:58:10.730424 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:10.725542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:41261 (local address 127.7.37.195:32847)
0926 14:58:10.725581 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:10.725583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:10.725748 (+ 165us) server_negotiation.cc:407] Connection header received
0926 14:58:10.727263 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:10.727265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:10.727277 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:10.727303 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:10.727708 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.728369 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.729560 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.730074 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.730117 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:10.730126 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:10.730322 (+ 196us) server_negotiation.cc:299] Negotiation successful
0926 14:58:10.730367 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:58:10.730435 20256 leader_election.cc:302] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62db0dcbc547494885b652fe360c21f3, 74c8ab1db5784e6faab3b44de24368a5; no voters:
I20260926 14:58:10.730551 20540 raft_consensus.cc:2781] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:58:10.730594 20540 raft_consensus.cc:482] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:58:10.730613 20540 raft_consensus.cc:3037] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:10.730690 20547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:10.725365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:32847 (local address 127.7.37.194:41261)
0926 14:58:10.725694 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:10.725703 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:10.727172 (+ 1469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:10.727360 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:10.727361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:10.727362 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:10.727420 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:10.727421 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.728459 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.728460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:10.729468 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:10.729471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.730180 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.730180 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:10.730245 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:10.730248 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:58:10.730584 (+ 336us) client_negotiation.cc:240] Negotiation successful
0926 14:58:10.730625 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":77,"threads_started":1}
I20260926 14:58:10.730965 20439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49182666b4bd4d32a29f73f32fb1e505" candidate_uuid: "62db0dcbc547494885b652fe360c21f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "930c76e3245e44e2913494cca530ccfa" is_pre_election: true
I20260926 14:58:10.731102 20439 raft_consensus.cc:2454] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62db0dcbc547494885b652fe360c21f3 in term 0.
I20260926 14:58:10.731463 20540 raft_consensus.cc:504] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "930c76e3245e44e2913494cca530ccfa" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 32847 } } peers { permanent_uuid: "74c8ab1db5784e6faab3b44de24368a5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 36887 } } peers { permanent_uuid: "62db0dcbc547494885b652fe360c21f3" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38453 } }
I20260926 14:58:10.731570 20540 leader_election.cc:288] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [CANDIDATE]: Term 1 election: Requested vote from peers 930c76e3245e44e2913494cca530ccfa (127.7.37.195:32847), 74c8ab1db5784e6faab3b44de24368a5 (127.7.37.193:36887)
I20260926 14:58:10.731777 20188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49182666b4bd4d32a29f73f32fb1e505" candidate_uuid: "62db0dcbc547494885b652fe360c21f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74c8ab1db5784e6faab3b44de24368a5"
I20260926 14:58:10.731783 20439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49182666b4bd4d32a29f73f32fb1e505" candidate_uuid: "62db0dcbc547494885b652fe360c21f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "930c76e3245e44e2913494cca530ccfa"
I20260926 14:58:10.731844 20188 raft_consensus.cc:3037] T 49182666b4bd4d32a29f73f32fb1e505 P 74c8ab1db5784e6faab3b44de24368a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:10.731844 20439 raft_consensus.cc:3037] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:10.732594 20188 raft_consensus.cc:2454] T 49182666b4bd4d32a29f73f32fb1e505 P 74c8ab1db5784e6faab3b44de24368a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62db0dcbc547494885b652fe360c21f3 in term 1.
I20260926 14:58:10.732594 20439 raft_consensus.cc:2454] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62db0dcbc547494885b652fe360c21f3 in term 1.
I20260926 14:58:10.732754 20255 leader_election.cc:302] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62db0dcbc547494885b652fe360c21f3, 930c76e3245e44e2913494cca530ccfa; no voters:
I20260926 14:58:10.732877 20540 raft_consensus.cc:2781] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:58:10.733006 20540 raft_consensus.cc:686] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [term 1 LEADER]: Becoming Leader. State: Replica: 62db0dcbc547494885b652fe360c21f3, State: Running, Role: LEADER
I20260926 14:58:10.733085 20540 consensus_queue.cc:237] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "930c76e3245e44e2913494cca530ccfa" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 32847 } } peers { permanent_uuid: "74c8ab1db5784e6faab3b44de24368a5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 36887 } } peers { permanent_uuid: "62db0dcbc547494885b652fe360c21f3" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38453 } }
I20260926 14:58:10.733762 20036 catalog_manager.cc:5495] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62db0dcbc547494885b652fe360c21f3 (127.7.37.194). New cstate: current_term: 1 leader_uuid: "62db0dcbc547494885b652fe360c21f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "930c76e3245e44e2913494cca530ccfa" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 32847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c8ab1db5784e6faab3b44de24368a5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62db0dcbc547494885b652fe360c21f3" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 38453 } health_report { overall_health: HEALTHY } } }
I20260926 14:58:10.741662 20533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:10.738486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49544 (local address 127.7.37.194:38453)
0926 14:58:10.738519 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:10.738521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:10.738528 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:58:10.738582 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:10.738583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:10.738599 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:10.738661 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:58:10.738804 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.739555 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.740663 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:10.740970 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:10.741025 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:10.741207 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:58:10.741319 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:58:10.741386 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:10.741583 (+ 197us) server_negotiation.cc:299] Negotiation successful
0926 14:58:10.741602 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 14:58:10.743556 20188 raft_consensus.cc:1261] T 49182666b4bd4d32a29f73f32fb1e505 P 74c8ab1db5784e6faab3b44de24368a5 [term 1 FOLLOWER]: Refusing update from remote peer 62db0dcbc547494885b652fe360c21f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:58:10.743834 20439 raft_consensus.cc:1261] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa [term 1 FOLLOWER]: Refusing update from remote peer 62db0dcbc547494885b652fe360c21f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:58:10.744135 20540 consensus_queue.cc:1035] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c8ab1db5784e6faab3b44de24368a5" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:58:10.744269 20540 consensus_queue.cc:1035] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "930c76e3245e44e2913494cca530ccfa" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 32847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:58:10.748167 20551 mvcc.cc:205] Tried to move back new op lower bound from 7333620493282947072 to 7333620493243084800. Current Snapshot: MvccSnapshot[applied={T|T < 7333620493282947072}]
I20260926 14:58:10.748705 20552 mvcc.cc:205] Tried to move back new op lower bound from 7333620493282947072 to 7333620493243084800. Current Snapshot: MvccSnapshot[applied={T|T < 7333620493282947072}]
I20260926 14:58:10.751991 20554 mvcc.cc:205] Tried to move back new op lower bound from 7333620493282947072 to 7333620493243084800. Current Snapshot: MvccSnapshot[applied={T|T < 7333620493282947072}]
I20260926 14:58:10.753046 20035 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:58:10.753135 20035 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:58:10.753789 20035 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55ccc7022b4d46d99ff94743d6e6929b: Sending DeleteTablet for 3 replicas of tablet 49182666b4bd4d32a29f73f32fb1e505
I20260926 14:58:10.754106 20418 tablet_service.cc:1514] Processing DeleteTablet for tablet 49182666b4bd4d32a29f73f32fb1e505 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:58:10 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:36110
I20260926 14:58:10.754473 20168 tablet_service.cc:1514] Processing DeleteTablet for tablet 49182666b4bd4d32a29f73f32fb1e505 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:58:10 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:59340
I20260926 14:58:10.754709 20295 tablet_service.cc:1514] Processing DeleteTablet for tablet 49182666b4bd4d32a29f73f32fb1e505 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:58:10 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:56086
I20260926 14:58:10.755122 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 20074
W20260926 14:58:10.767002 20021 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.37.193:36887 (error 108)
I20260926 14:58:10.767153 20560 tablet_replica.cc:330] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa: stopping tablet replica
I20260926 14:58:10.767269 20560 raft_consensus.cc:2229] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:58:10.767366 20560 raft_consensus.cc:2258] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:58:10.767504 20562 tablet_replica.cc:330] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3: stopping tablet replica
I20260926 14:58:10.767585 20562 raft_consensus.cc:2229] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:58:10.767657 20562 raft_consensus.cc:2258] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:58:10.767938 20560 ts_tablet_manager.cc:1905] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:58:10.768060 20562 ts_tablet_manager.cc:1905] T 49182666b4bd4d32a29f73f32fb1e505 P 62db0dcbc547494885b652fe360c21f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:58:10.771194 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 20238
I20260926 14:58:10.771795 20560 ts_tablet_manager.cc:1918] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:58:10.771872 20560 log.cc:1196] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/49182666b4bd4d32a29f73f32fb1e505
I20260926 14:58:10.772101 20560 ts_tablet_manager.cc:1939] T 49182666b4bd4d32a29f73f32fb1e505 P 930c76e3245e44e2913494cca530ccfa: Deleting consensus metadata
I20260926 14:58:10.772470 20020 catalog_manager.cc:4841] TS 930c76e3245e44e2913494cca530ccfa (127.7.37.195:32847): tablet 49182666b4bd4d32a29f73f32fb1e505 (table test-table [id=8c2f30f76ebc47d08f5bd8b42c01698b]) successfully deleted
I20260926 14:58:10.776528 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 20371
I20260926 14:58:10.781045 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 20003
I20260926 14:58:10.785341 7319 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 14:58:11.118595 7319 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 14:58:11.118680 7319 mini_ranger.cc:66] Stopping Ranger...
I20260926 14:58:11.451799 7319 mini_ranger.cc:68] Stopped Ranger
2026-09-26 14:58:11.451 UTC [19070] LOG: received smart shutdown request
2026-09-26 14:58:11.455 UTC [19070] LOG: background worker "logical replication launcher" (PID 19079) exited with exit code 1
2026-09-26 14:58:11.455 UTC [19074] LOG: shutting down
2026-09-26 14:58:11.590 UTC [19070] LOG: database system is shut down
2026-09-26T14:58:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (42859 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:58:11 dist-test-slave-f9xb krb5kdc[20572](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:58:11 dist-test-slave-f9xb krb5kdc[20572](info): set up 2 sockets
Sep 26 14:58:11 dist-test-slave-f9xb krb5kdc[20572](info): commencing operation
krb5kdc: starting...
W20260926 14:58:12.615233 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.920s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:58:12 dist-test-slave-f9xb krb5kdc[20572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:58:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:58:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:58:12.717567 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:39227
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:41905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:39227,127.7.37.253:37137,127.7.37.252:43069
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:58:12.794095 20585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:12.794296 20585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:12.794327 20585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:12.795431 20585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:58:12.795467 20585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:12.795486 20585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:12.795501 20585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:58:12.795516 20585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:58:12.796967 20585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:41905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:39227,127.7.37.253:37137,127.7.37.252:43069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:39227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:12.797170 20585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:12.797348 20585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:12.801594 20591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:12.801599 20590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:12.801599 20593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:12.801787 20585 server_base.cc:1034] running on GCE node
I20260926 14:58:12.802000 20585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:12.802261 20585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:12.803418 20585 hybrid_clock.cc:648] HybridClock initialized: now 1790434692803400 us; error 28 us; skew 500 ppm
Sep 26 14:58:12 dist-test-slave-f9xb krb5kdc[20572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434692, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:12.809127 20585 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:58:12.809504 20585 webserver.cc:460] Webserver started at http://127.7.37.254:34449/ using document root <none> and password file <none>
I20260926 14:58:12.809693 20585 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:12.809738 20585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:12.809851 20585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:58:12.810775 20585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30"
format_stamp: "Formatted at 2026-09-26 14:58:12 on dist-test-slave-f9xb"
server_key: "85ad9cf0dc73779e9e1b39ffd5982397"
server_key_iv: "d2d83b733fb16dddc6b9d9a360ee14a9"
server_key_version: "encryptionkey@0"
I20260926 14:58:12.811126 20585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30"
format_stamp: "Formatted at 2026-09-26 14:58:12 on dist-test-slave-f9xb"
server_key: "85ad9cf0dc73779e9e1b39ffd5982397"
server_key_iv: "d2d83b733fb16dddc6b9d9a360ee14a9"
server_key_version: "encryptionkey@0"
I20260926 14:58:12.812314 20585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:58:12.812918 20600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:12.813057 20585 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:12.813143 20585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30"
format_stamp: "Formatted at 2026-09-26 14:58:12 on dist-test-slave-f9xb"
server_key: "85ad9cf0dc73779e9e1b39ffd5982397"
server_key_iv: "d2d83b733fb16dddc6b9d9a360ee14a9"
server_key_version: "encryptionkey@0"
I20260926 14:58:12.813200 20585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:12.829888 20585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:12.831382 20585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:12.831496 20585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:12.834839 20585 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:39227
I20260926 14:58:12.834870 20652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:39227 every 8 connection(s)
I20260926 14:58:12.835145 20585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:58:12.835850 20653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:12.837136 20653 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } has no permanent_uuid. Determining permanent_uuid...
I20260926 14:58:12.841277 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 20585
I20260926 14:58:12.841372 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:58:12.841542 7319 external_mini_cluster.cc:1428] Setting key af87b6daf6595db4b43113d5ffb209bd
Sep 26 14:58:12 dist-test-slave-f9xb krb5kdc[20572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434692, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:12.845350 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:12.837669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:39227 (local address 127.0.0.1:36598)
0926 14:58:12.837810 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:12.837839 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:12.838220 (+ 381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:12.838428 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:12.838430 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:12.838636 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:12.838773 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:12.838776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:12.839692 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:12.839693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:12.840353 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:12.840355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:12.840771 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:12.840771 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:12.840821 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:12.841157 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:12.841164 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:12.842453 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:12.844122 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:12.844128 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:12.844129 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:12.844298 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:12.844531 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:12.844532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:12.844532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:12.844623 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:12.845006 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:12.845009 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:12.845053 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 14:58:12.845112 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 14:58:12.845216 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260926 14:58:12.845419 20656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:12.837693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36598 (local address 127.7.37.254:39227)
0926 14:58:12.837835 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:12.837843 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:12.837920 (+ 77us) server_negotiation.cc:407] Connection header received
0926 14:58:12.838316 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:12.838317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:12.838333 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:12.838418 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:12.838832 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:12.839617 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:12.840420 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:12.840716 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:12.840785 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:12.842541 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:12.842549 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:12.842549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:12.842556 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:12.844054 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:12.844367 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:12.844369 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:12.844370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:12.844463 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:12.844679 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:12.844679 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:12.844680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:12.844942 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:12.845015 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:12.845156 (+ 141us) server_negotiation.cc:299] Negotiation successful
0926 14:58:12.845294 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
I20260926 14:58:12.846184 20653 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } has no permanent_uuid. Determining permanent_uuid...
I20260926 14:58:12.846441 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:12.846350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:37137 (local address 127.0.0.1:45672)
0926 14:58:12.846386 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:12.846424 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.253:37137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260926 14:58:12.846553 20603 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.37.253:37137: connect: Connection refused (error 111)
I20260926 14:58:12.846889 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:12.846807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:37137 (local address 127.0.0.1:45676)
0926 14:58:12.846839 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:12.846862 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.253:37137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260926 14:58:12.846973 20653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.253:37137: Network error: Client connection negotiation failed: client connection to 127.7.37.253:37137: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.7.37.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.253.keytab.
Entry for principal kudu/127.7.37.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.253.keytab.
I20260926 14:58:12.861918 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.253:37137
--webserver_interface=127.7.37.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.253.keytab
--principal=kudu/127.7.37.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:41905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:39227,127.7.37.253:37137,127.7.37.252:43069
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 14:58:12.896150 20653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } attempt: 1
I20260926 14:58:12.896468 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:12.896343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:37137 (local address 127.0.0.1:45678)
0926 14:58:12.896405 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:12.896440 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.253:37137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 14:58:12.896744 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:12.896663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:37137 (local address 127.0.0.1:45684)
0926 14:58:12.896701 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:12.896727 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.253:37137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260926 14:58:12.896833 20653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.253:37137: Network error: Client connection negotiation failed: client connection to 127.7.37.253:37137: connect: Connection refused (error 111)
W20260926 14:58:12.946556 20660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:12.946722 20660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:12.946741 20660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:12.947865 20660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:58:12.947901 20660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:12.947913 20660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:12.947935 20660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:58:12.947957 20660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:58:12.949342 20660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:41905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:39227,127.7.37.253:37137,127.7.37.252:43069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.253.keytab
--principal=kudu/127.7.37.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.253:37137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.37.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:12.949543 20660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:12.949723 20660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:12.953972 20666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:12.953982 20668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:12.953982 20665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:12.954160 20660 server_base.cc:1034] running on GCE node
I20260926 14:58:12.954365 20660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:12.954627 20660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:12.955770 20660 hybrid_clock.cc:648] HybridClock initialized: now 1790434692955751 us; error 28 us; skew 500 ppm
Sep 26 14:58:12 dist-test-slave-f9xb krb5kdc[20572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434692, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:12.957859 20660 init.cc:377] Logged in from keytab as kudu/127.7.37.253@KRBTEST.COM (short username kudu)
I20260926 14:58:12.958221 20660 webserver.cc:460] Webserver started at http://127.7.37.253:42975/ using document root <none> and password file <none>
I20260926 14:58:12.958392 20660 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:12.958436 20660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:12.958554 20660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:58:12.959440 20660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-1/data/instance:
uuid: "d743b8f143c642c2b2d80f01ff752cbd"
format_stamp: "Formatted at 2026-09-26 14:58:12 on dist-test-slave-f9xb"
server_key: "b88d2aadc05e4cc066530cf3fd942053"
server_key_iv: "1d67723927cc568219fd2639ba3cc53a"
server_key_version: "encryptionkey@0"
I20260926 14:58:12.959764 20660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-1/wal/instance:
uuid: "d743b8f143c642c2b2d80f01ff752cbd"
format_stamp: "Formatted at 2026-09-26 14:58:12 on dist-test-slave-f9xb"
server_key: "b88d2aadc05e4cc066530cf3fd942053"
server_key_iv: "1d67723927cc568219fd2639ba3cc53a"
server_key_version: "encryptionkey@0"
I20260926 14:58:12.960881 20660 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:58:12.961459 20675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:12.961593 20660 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:58:12.961668 20660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-1/wal
uuid: "d743b8f143c642c2b2d80f01ff752cbd"
format_stamp: "Formatted at 2026-09-26 14:58:12 on dist-test-slave-f9xb"
server_key: "b88d2aadc05e4cc066530cf3fd942053"
server_key_iv: "1d67723927cc568219fd2639ba3cc53a"
server_key_version: "encryptionkey@0"
I20260926 14:58:12.961725 20660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:12.964988 20653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } attempt: 2
I20260926 14:58:12.965313 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:12.965203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:37137 (local address 127.0.0.1:45694)
0926 14:58:12.965245 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:12.965280 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.253:37137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260926 14:58:12.965613 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:12.965545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:37137 (local address 127.0.0.1:45710)
0926 14:58:12.965575 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:12.965597 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.253:37137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260926 14:58:12.965690 20653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.253:37137: Network error: Client connection negotiation failed: client connection to 127.7.37.253:37137: connect: Connection refused (error 111)
I20260926 14:58:12.973914 20660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:12.975437 20660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:12.975548 20660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:12.978919 20660 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.253:37137
I20260926 14:58:12.978955 20727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.253:37137 every 8 connection(s)
I20260926 14:58:12.979209 20660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-1/data/info.pb
I20260926 14:58:12.979923 20728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:12.981165 20728 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } has no permanent_uuid. Determining permanent_uuid...
I20260926 14:58:12.985510 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 20660
I20260926 14:58:12.985606 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-1/wal/instance
I20260926 14:58:12.985751 7319 external_mini_cluster.cc:1428] Setting key 92a70087ea7466ea4c7926d9d7be0a79
Sep 26 14:58:12 dist-test-slave-f9xb krb5kdc[20572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434692, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.253@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:12.988662 20656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:12.981684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36608 (local address 127.7.37.254:39227)
0926 14:58:12.981742 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:12.981744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:12.981904 (+ 160us) server_negotiation.cc:407] Connection header received
0926 14:58:12.982273 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:12.982273 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:12.982284 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:12.982327 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:12.982817 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:12.983273 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:12.984171 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:12.984360 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:12.984407 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:12.986206 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:12.986208 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:12.986208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:12.986215 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:12.987398 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:12.987861 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:12.987862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:12.987863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:12.987933 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:12.988135 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:12.988136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:12.988136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:12.988256 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:12.988316 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:12.988511 (+ 195us) server_negotiation.cc:299] Negotiation successful
0926 14:58:12.988586 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 14:58:12.988716 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:12.981668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:39227 (local address 127.0.0.1:36608)
0926 14:58:12.981811 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:12.981840 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:12.982209 (+ 369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:12.982373 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:12.982377 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:12.982575 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:12.982742 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:12.982745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:12.983327 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:12.983327 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:12.984118 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:12.984120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:12.984443 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:12.984444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:12.984543 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:12.984900 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:12.984907 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:12.986126 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:12.987470 (+ 1344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:12.987476 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:12.987478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:12.987781 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:12.987995 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:12.987995 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:12.987996 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:12.988086 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:12.988311 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:12.988313 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:12.988410 (+ 97us) client_negotiation.cc:769] Sending connection context
0926 14:58:12.988482 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 14:58:12.988581 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
I20260926 14:58:12.989212 20728 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.37.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.252@KRBTEST.COM" created.
Sep 26 14:58:12 dist-test-slave-f9xb krb5kdc[20572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434692, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.253@KRBTEST.COM for kudu/127.7.37.253@KRBTEST.COM
I20260926 14:58:12.998054 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:12.989401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:37137 (local address 127.0.0.1:45720)
0926 14:58:12.989438 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:12.989448 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:12.989484 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:12.989717 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:12.989718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:12.989821 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:12.989863 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:12.989864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:12.991050 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:12.991051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:12.991528 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:12.991530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:12.992117 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:12.992118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:12.992169 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:12.992480 (+ 311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:12.992484 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:12.995456 (+ 2972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:12.997091 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:12.997093 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:12.997093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:12.997281 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:12.997502 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:12.997503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:12.997503 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:12.997558 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:12.997770 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:12.997770 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:12.997810 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 14:58:12.997877 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 14:58:12.997961 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260926 14:58:12.998106 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:12.989435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45720 (local address 127.7.37.253:37137)
0926 14:58:12.989564 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:12.989568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:12.989587 (+ 19us) server_negotiation.cc:407] Connection header received
0926 14:58:12.989607 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:12.989609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:12.989621 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:12.989697 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:12.989931 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:12.990960 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:12.991596 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:12.992052 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:12.992124 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:12.995584 (+ 3460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:12.995591 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:12.995591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:12.995597 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:12.997024 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:12.997385 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:12.997387 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:12.997388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:12.997440 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:12.997610 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:12.997610 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:12.997611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:12.997721 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:12.997775 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:12.997899 (+ 124us) server_negotiation.cc:299] Negotiation successful
0926 14:58:12.998008 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
I20260926 14:58:12.998582 20728 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } has no permanent_uuid. Determining permanent_uuid...
I20260926 14:58:12.998864 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:12.998760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:43069 (local address 127.0.0.1:43416)
0926 14:58:12.998802 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:12.998844 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:43069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 14:58:12.998993 20679 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.37.252:43069: connect: Connection refused (error 111)
I20260926 14:58:12.999356 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:12.999255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:43069 (local address 127.0.0.1:43418)
0926 14:58:12.999309 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:12.999335 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:43069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 14:58:12.999452 20728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.252:43069: Network error: Client connection negotiation failed: client connection to 127.7.37.252:43069: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.252.keytab.
Entry for principal kudu/127.7.37.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.252.keytab.
I20260926 14:58:13.010124 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.252:43069
--webserver_interface=127.7.37.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.252.keytab
--principal=kudu/127.7.37.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:41905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:39227,127.7.37.253:37137,127.7.37.252:43069
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 14:58:13.048657 20728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } attempt: 1
I20260926 14:58:13.049153 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.048992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:43069 (local address 127.0.0.1:43424)
0926 14:58:13.049059 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:13.049111 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:43069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 14:58:13.049696 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.049536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:43069 (local address 127.0.0.1:43434)
0926 14:58:13.049649 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:13.049675 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:43069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260926 14:58:13.049887 20728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.252:43069: Network error: Client connection negotiation failed: client connection to 127.7.37.252:43069: connect: Connection refused (error 111)
I20260926 14:58:13.056829 20653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } attempt: 3
Sep 26 14:58:13 dist-test-slave-f9xb krb5kdc[20572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434692, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for kudu/127.7.37.253@KRBTEST.COM
I20260926 14:58:13.065623 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.057171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45722 (local address 127.7.37.253:37137)
0926 14:58:13.057209 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:13.057210 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:13.057223 (+ 13us) server_negotiation.cc:407] Connection header received
0926 14:58:13.057292 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:13.057293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:13.057304 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:13.057350 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:13.057678 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.058194 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.059143 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.059341 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.059388 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:13.061623 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:13.061624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:13.061625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:13.061631 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:13.064464 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.064870 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.064871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.064872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.064930 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.065103 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.065104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.065104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.065224 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:13.065285 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:13.065466 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 14:58:13.065540 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 14:58:13.065644 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.057121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:37137 (local address 127.0.0.1:45722)
0926 14:58:13.057170 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:13.057180 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:13.057220 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:13.057367 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:13.057368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:13.057539 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:13.057610 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:13.057612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.058263 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.058264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:13.059067 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:13.059070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.059399 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.059400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:13.059480 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:13.060055 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:13.060060 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:13.061527 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:13.064556 (+ 3029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:13.064556 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:13.064557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:13.064788 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:13.064993 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:13.064994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:13.064994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:13.065049 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:13.065285 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:13.065287 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:13.065347 (+ 60us) client_negotiation.cc:769] Sending connection context
0926 14:58:13.065440 (+ 93us) client_negotiation.cc:240] Negotiation successful
0926 14:58:13.065548 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 14:58:13.066394 20653 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } has no permanent_uuid. Determining permanent_uuid...
I20260926 14:58:13.066749 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.066642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:43069 (local address 127.0.0.1:43448)
0926 14:58:13.066689 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:13.066723 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:43069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260926 14:58:13.067138 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.066973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:43069 (local address 127.0.0.1:43452)
0926 14:58:13.067084 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:13.067110 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:43069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
W20260926 14:58:13.067211 20653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.252:43069: Network error: Client connection negotiation failed: client connection to 127.7.37.252:43069: connect: Connection refused (error 111)
I20260926 14:58:13.098321 20653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } attempt: 1
I20260926 14:58:13.098685 20736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.098551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:43069 (local address 127.0.0.1:43454)
0926 14:58:13.098609 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:13.098649 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:43069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260926 14:58:13.099013 20736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.098920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:43069 (local address 127.0.0.1:43460)
0926 14:58:13.098949 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:13.098975 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:43069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260926 14:58:13.099099 20653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.252:43069: Network error: Client connection negotiation failed: client connection to 127.7.37.252:43069: connect: Connection refused (error 111)
W20260926 14:58:13.100294 20735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:13.100466 20735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:13.100495 20735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:13.101692 20735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:58:13.101732 20735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:13.101750 20735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:13.101768 20735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:58:13.101790 20735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:58:13.103231 20735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:41905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:39227,127.7.37.253:37137,127.7.37.252:43069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.252.keytab
--principal=kudu/127.7.37.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.252:43069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.37.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:13.103453 20735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:13.103626 20735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:13.107897 20742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:13.107903 20741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:13.108083 20735 server_base.cc:1034] running on GCE node
W20260926 14:58:13.108024 20744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:13.108503 20735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:13.108804 20735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:13.109941 20735 hybrid_clock.cc:648] HybridClock initialized: now 1790434693109918 us; error 37 us; skew 500 ppm
Sep 26 14:58:13 dist-test-slave-f9xb krb5kdc[20572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434693, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:13.112316 20735 init.cc:377] Logged in from keytab as kudu/127.7.37.252@KRBTEST.COM (short username kudu)
I20260926 14:58:13.112673 20735 webserver.cc:460] Webserver started at http://127.7.37.252:43525/ using document root <none> and password file <none>
I20260926 14:58:13.112886 20735 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:13.112943 20735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:13.113058 20735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:58:13.113960 20735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-2/data/instance:
uuid: "0a2c87156a2446bbb8cde25bf31b098f"
format_stamp: "Formatted at 2026-09-26 14:58:13 on dist-test-slave-f9xb"
server_key: "e3c0c959ec059ab50705130a5d2ce0ca"
server_key_iv: "6bf042750aa56dc881b4dee56882ae12"
server_key_version: "encryptionkey@0"
I20260926 14:58:13.114249 20735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-2/wal/instance:
uuid: "0a2c87156a2446bbb8cde25bf31b098f"
format_stamp: "Formatted at 2026-09-26 14:58:13 on dist-test-slave-f9xb"
server_key: "e3c0c959ec059ab50705130a5d2ce0ca"
server_key_iv: "6bf042750aa56dc881b4dee56882ae12"
server_key_version: "encryptionkey@0"
I20260926 14:58:13.115440 20735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:58:13.116124 20751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:13.116281 20735 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:13.116356 20735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-2/wal
uuid: "0a2c87156a2446bbb8cde25bf31b098f"
format_stamp: "Formatted at 2026-09-26 14:58:13 on dist-test-slave-f9xb"
server_key: "e3c0c959ec059ab50705130a5d2ce0ca"
server_key_iv: "6bf042750aa56dc881b4dee56882ae12"
server_key_version: "encryptionkey@0"
I20260926 14:58:13.116413 20735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:13.118044 20728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } attempt: 2
I20260926 14:58:13.118396 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.118280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:43069 (local address 127.0.0.1:43464)
0926 14:58:13.118338 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:13.118369 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:43069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 14:58:13.118691 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.118609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:43069 (local address 127.0.0.1:43478)
0926 14:58:13.118643 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:13.118667 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:43069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260926 14:58:13.118803 20728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.252:43069: Network error: Client connection negotiation failed: client connection to 127.7.37.252:43069: connect: Connection refused (error 111)
I20260926 14:58:13.128366 20735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:13.129863 20735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:13.129958 20735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:13.133391 20735 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.252:43069
I20260926 14:58:13.133436 20803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.252:43069 every 8 connection(s)
I20260926 14:58:13.133692 20735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-2/data/info.pb
I20260926 14:58:13.134559 20804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:13.134944 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 20735
I20260926 14:58:13.135073 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434555014799-7319-0/minicluster-data/master-2/wal/instance
I20260926 14:58:13.135249 7319 external_mini_cluster.cc:1428] Setting key c9eae373c62fb09f2d2f39207706cae0
I20260926 14:58:13.136274 20804 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 14:58:13 dist-test-slave-f9xb krb5kdc[20572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434693, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.252@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
Sep 26 14:58:13 dist-test-slave-f9xb krb5kdc[20572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.252@KRBTEST.COM
I20260926 14:58:13.144148 20656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.136858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36624 (local address 127.7.37.254:39227)
0926 14:58:13.136906 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:13.136908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:13.136977 (+ 69us) server_negotiation.cc:407] Connection header received
0926 14:58:13.137034 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:13.137035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:13.137044 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:13.137092 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:13.137615 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.138643 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.139603 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.139889 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.139947 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:13.141674 (+ 1727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:13.141676 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:13.141676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:13.141683 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:13.142862 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.143236 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.143237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.143237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.143289 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.143510 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.143511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.143512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.143626 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:13.143690 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:13.143988 (+ 298us) server_negotiation.cc:299] Negotiation successful
0926 14:58:13.144062 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 14:58:13.144289 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.136779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:39227 (local address 127.0.0.1:36624)
0926 14:58:13.136895 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:13.136929 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:13.136991 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:13.137108 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:13.137110 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:13.137411 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:13.137539 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:13.137544 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.138727 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.138728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:13.139519 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:13.139521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.139965 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.139966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:13.140039 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:13.140400 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:13.140407 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:13.141602 (+ 1195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:13.142931 (+ 1329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:13.142937 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:13.142938 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:13.143176 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:13.143366 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:13.143367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:13.143367 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:13.143455 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:13.143697 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:13.143702 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:13.143857 (+ 155us) client_negotiation.cc:769] Sending connection context
0926 14:58:13.143962 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 14:58:13.144092 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":3,"thread_start_us":69,"threads_started":1}
I20260926 14:58:13.144728 20804 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } has no permanent_uuid. Determining permanent_uuid...
I20260926 14:58:13.144831 20807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.135699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43482 (local address 127.7.37.252:43069)
0926 14:58:13.135894 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:13.135903 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:13.135927 (+ 24us) server_negotiation.cc:407] Connection header received
0926 14:58:13.136296 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:13.136302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:13.136347 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:13.136462 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:13.137085 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.138098 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.139003 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.139544 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.139635 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:13.142168 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:13.142173 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:13.142174 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:13.142179 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:13.143557 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.143893 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.143896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.143896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.143951 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.144134 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.144135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.144135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.144274 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:13.144640 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:13.144698 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 14:58:13.144764 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":114,"threads_started":1}
Sep 26 14:58:13 dist-test-slave-f9xb krb5kdc[20572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434693, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.252@KRBTEST.COM for kudu/127.7.37.253@KRBTEST.COM
Sep 26 14:58:13 dist-test-slave-f9xb krb5kdc[20572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:13.154001 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.145004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45724 (local address 127.7.37.253:37137)
0926 14:58:13.145043 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:13.145045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:13.145071 (+ 26us) server_negotiation.cc:407] Connection header received
0926 14:58:13.145127 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:13.145128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:13.145136 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:13.145188 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:13.145485 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.146160 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.146897 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.147089 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.147136 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:13.148925 (+ 1789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:13.148926 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:13.148927 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:13.148934 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:13.151670 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.153081 (+ 1411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.153083 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.153083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.153159 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.153439 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.153440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.153440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.153575 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:13.153616 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:13.153834 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 14:58:13.153911 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 14:58:13.154249 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.144932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:37137 (local address 127.0.0.1:45724)
0926 14:58:13.145030 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:13.145039 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:13.145083 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:13.145203 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:13.145204 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:13.145359 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:13.145391 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:13.145392 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.146227 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.146227 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:13.146814 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:13.146827 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.147146 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.147147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:13.147197 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:13.147528 (+ 331us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:13.147532 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:13.148841 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:13.152761 (+ 3920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:13.152762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:13.152763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:13.152988 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:13.153251 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:13.153252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:13.153252 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:13.153345 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:13.153663 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:13.153664 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:13.153704 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 14:58:13.154082 (+ 378us) client_negotiation.cc:240] Negotiation successful
0926 14:58:13.154167 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260926 14:58:13.154589 20656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.145444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36628 (local address 127.7.37.254:39227)
0926 14:58:13.145473 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:13.145475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:13.145488 (+ 13us) server_negotiation.cc:407] Connection header received
0926 14:58:13.145528 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:13.145528 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:13.145534 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:13.145573 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:13.146044 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.146451 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.147201 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.147369 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.147419 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:13.149953 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:13.149954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:13.149955 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:13.149961 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:13.153306 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.153688 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.153689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.153691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.153745 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.153936 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.153937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.153938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.154054 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:13.154213 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:13.154421 (+ 208us) server_negotiation.cc:299] Negotiation successful
0926 14:58:13.154503 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260926 14:58:13.154731 20804 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 14:58:13 dist-test-slave-f9xb krb5kdc[20572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434693, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.252@KRBTEST.COM for kudu/127.7.37.252@KRBTEST.COM
Sep 26 14:58:13 dist-test-slave-f9xb krb5kdc[20572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.253@KRBTEST.COM
I20260926 14:58:13.161988 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.155370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45740 (local address 127.7.37.253:37137)
0926 14:58:13.155421 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:13.155423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:13.155449 (+ 26us) server_negotiation.cc:407] Connection header received
0926 14:58:13.155523 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:13.155523 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:13.155533 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:13.155589 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:13.156134 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.156724 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.157449 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.157684 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.157715 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:13.159859 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:13.159860 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:13.159860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:13.159867 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:13.161059 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.161361 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.161362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.161362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.161414 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.161566 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.161566 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.161567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.161664 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:13.161808 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:13.161842 (+ 34us) server_negotiation.cc:299] Negotiation successful
0926 14:58:13.161906 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 14:58:13.162113 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.155011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:43069 (local address 127.0.0.1:43488)
0926 14:58:13.155048 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:13.155098 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:13.155132 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:13.155462 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:13.155463 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:13.155592 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:13.155618 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:13.155620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.156464 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.156465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:13.157149 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:13.157152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.157572 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.157572 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:13.157614 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:13.157966 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:13.157970 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:13.159713 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:13.161300 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:13.161301 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:13.161301 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:13.161436 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:13.161597 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:13.161598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:13.161598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:13.161647 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:13.161839 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:13.161840 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:13.161874 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 14:58:13.161933 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 14:58:13.162018 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260926 14:58:13.162164 20807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.155080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43488 (local address 127.7.37.252:43069)
0926 14:58:13.155355 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:13.155357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:13.155364 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:58:13.155379 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:13.155380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:13.155388 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:13.155433 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:13.155677 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.156381 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.157229 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.157485 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.157533 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:13.159844 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:13.159845 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:13.159846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:13.159852 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:13.161227 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.161501 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.161502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.161502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.161548 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.161694 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.161694 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.161695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.161785 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:13.161835 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:13.161968 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 14:58:13.162064 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":260}
I20260926 14:58:13.163156 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:58:13.163407 20804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f: Bootstrap starting.
I20260926 14:58:13.164245 20804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:13.164546 20804 log.cc:824] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:13.165270 20804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f: No bootstrap required, opened a new log
I20260926 14:58:13.166976 20804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } } peers { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } } peers { permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } }
I20260926 14:58:13.167114 20804 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:13.167146 20804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2c87156a2446bbb8cde25bf31b098f, State: Initialized, Role: FOLLOWER
I20260926 14:58:13.167253 20804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } } peers { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } } peers { permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } }
I20260926 14:58:13.167476 20817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } } peers { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } } peers { permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } } }
I20260926 14:58:13.167533 20804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:58:13.167562 20817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f [sys.catalog]: This master's current role is: FOLLOWER
W20260926 14:58:13.169312 20828 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 14:58:13.169410 20828 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 14:58:13.169590 20735 master_runner.cc:186] Error getting master registration for 127.7.37.254:39227: OK, instance_id { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" instance_seqno: 1790434692833947 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 14:58:13.171736 20735 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 14:58:13.172710 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.164459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45748 (local address 127.7.37.253:37137)
0926 14:58:13.164517 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:13.164518 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:13.164916 (+ 398us) server_negotiation.cc:407] Connection header received
0926 14:58:13.165130 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:13.165131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:13.165140 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:13.165166 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:13.165815 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.166477 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.167087 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.167301 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.167354 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:13.168667 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:13.168668 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:13.168669 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:13.168675 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:13.171372 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.171667 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.171668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.171668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.171723 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.171997 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.171998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.171999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.172119 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:13.172199 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:13.172560 (+ 361us) server_negotiation.cc:299] Negotiation successful
0926 14:58:13.172632 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 14:58:13.173451 20807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.164678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43492 (local address 127.7.37.252:43069)
0926 14:58:13.164816 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:13.164818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:13.165005 (+ 187us) server_negotiation.cc:407] Connection header received
0926 14:58:13.165057 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:13.165057 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:13.165066 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:13.165115 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:13.166573 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.167406 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.168387 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.168654 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.168976 (+ 322us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:13.170407 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:13.170408 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:13.170409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:13.170415 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:13.172057 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.172365 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.172366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.172367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.172418 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.172625 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.172626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.172626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.173009 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:13.173309 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:13.173345 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 14:58:13.173402 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260926 14:58:13.174226 20653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } attempt: 2
I20260926 14:58:13.175832 20656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.164598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36632 (local address 127.7.37.254:39227)
0926 14:58:13.164630 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:13.164631 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:13.164883 (+ 252us) server_negotiation.cc:407] Connection header received
0926 14:58:13.168723 (+ 3840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:13.168724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:13.168733 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:13.168783 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:13.169367 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.169999 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.170915 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.171186 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.171241 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:13.173122 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:13.173123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:13.173123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:13.173129 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:13.174682 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.175057 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.175058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.175059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.175111 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.175283 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.175284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.175284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.175392 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:13.175466 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:13.175651 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 14:58:13.175718 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Sep 26 14:58:13 dist-test-slave-f9xb krb5kdc[20572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434692, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for kudu/127.7.37.252@KRBTEST.COM
I20260926 14:58:13.180219 20807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.174588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43494 (local address 127.7.37.252:43069)
0926 14:58:13.174628 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:13.174629 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:13.174635 (+ 6us) server_negotiation.cc:407] Connection header received
0926 14:58:13.174650 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:13.174651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:13.174658 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:13.174701 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:13.174953 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.175357 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.176182 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.176435 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.176504 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:13.178223 (+ 1719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:13.178223 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:13.178224 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:13.178230 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:13.179328 (+ 1098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.179598 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.179600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.179600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.179646 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.179813 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.179814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.179814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.179906 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:13.179955 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:13.180083 (+ 128us) server_negotiation.cc:299] Negotiation successful
0926 14:58:13.180140 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 14:58:13.180226 20736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.174439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:43069 (local address 127.0.0.1:43494)
0926 14:58:13.174484 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:13.174494 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:13.174538 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:13.174714 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:13.174714 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:13.174849 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:13.174893 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:13.174895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.175417 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.175418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:13.176114 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:13.176117 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.176507 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.176508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:13.176580 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:13.176904 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:13.176908 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:13.178158 (+ 1250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:13.179397 (+ 1239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:13.179398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:13.179398 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:13.179545 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:13.179696 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:13.179697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:13.179697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:13.179770 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:13.179954 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:13.179955 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:13.180001 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 14:58:13.180056 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 14:58:13.180141 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260926 14:58:13.181294 20653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30: Bootstrap starting.
I20260926 14:58:13.181818 20653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:13.182075 20653 log.cc:824] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:13.182701 20653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30: No bootstrap required, opened a new log
I20260926 14:58:13.184248 20653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } } peers { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } } peers { permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } }
I20260926 14:58:13.184345 20653 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:13.184365 20653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87b895c02e4a46ee8b1ad0f9f72cfe30, State: Initialized, Role: FOLLOWER
I20260926 14:58:13.184454 20653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } } peers { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } } peers { permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } }
I20260926 14:58:13.184686 20653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:58:13.184666 20831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } } peers { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } } peers { permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } } }
I20260926 14:58:13.184825 20831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 14:58:13.186347 20842 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 14:58:13.186395 20842 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 14:58:13.186669 20585 master_runner.cc:186] Error getting master registration for 127.7.37.253:37137: OK, instance_id { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" instance_seqno: 1790434692977990 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 14:58:13.188416 20585 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 14:58:13.209950 20728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } attempt: 3
Sep 26 14:58:13 dist-test-slave-f9xb krb5kdc[20572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434692, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.253@KRBTEST.COM for kudu/127.7.37.252@KRBTEST.COM
I20260926 14:58:13.215322 20807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.210192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43498 (local address 127.7.37.252:43069)
0926 14:58:13.210223 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:13.210225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:13.210232 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:58:13.210284 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:13.210284 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:13.210293 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:13.210335 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:13.210572 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.210992 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.211586 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.211789 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.211840 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:13.213304 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:13.213306 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:13.213306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:13.213312 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:13.214437 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.214714 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.214715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.214715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.214762 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:13.214909 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:13.214910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:13.214910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:13.215007 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:13.215057 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:13.215183 (+ 126us) server_negotiation.cc:299] Negotiation successful
0926 14:58:13.215240 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 14:58:13.215342 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:13.210154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:43069 (local address 127.0.0.1:43498)
0926 14:58:13.210197 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:13.210206 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:13.210240 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:13.210342 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:13.210343 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:13.210464 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:13.210510 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:13.210511 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.211052 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.211053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:13.211521 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:13.211522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:13.211841 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:13.211842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:13.211893 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:13.212227 (+ 334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:13.212231 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:13.213235 (+ 1004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:13.214508 (+ 1273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:13.214509 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:13.214509 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:13.214660 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:13.214810 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:13.214810 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:13.214811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:13.214862 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:13.215056 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:13.215057 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:13.215098 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 14:58:13.215156 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 14:58:13.215240 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 14:58:13.216792 20728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd: Bootstrap starting.
I20260926 14:58:13.217316 20728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:13.217592 20728 log.cc:824] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:13.218469 20728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd: No bootstrap required, opened a new log
I20260926 14:58:13.219965 20728 raft_consensus.cc:348] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } } peers { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } } peers { permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } }
I20260926 14:58:13.220069 20728 raft_consensus.cc:374] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:13.220100 20728 raft_consensus.cc:729] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d743b8f143c642c2b2d80f01ff752cbd, State: Initialized, Role: FOLLOWER
I20260926 14:58:13.220206 20728 consensus_queue.cc:260] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } } peers { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } } peers { permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } }
I20260926 14:58:13.220410 20845 sys_catalog.cc:455] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } } peers { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } } peers { permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } } }
I20260926 14:58:13.220464 20728 sys_catalog.cc:564] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:58:13.220521 20845 sys_catalog.cc:458] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd [sys.catalog]: This master's current role is: FOLLOWER
W20260926 14:58:13.222028 20856 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 14:58:13.222121 20856 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 14:58:13.223934 20660 master_runner.cc:263] No leader master found from master d743b8f143c642c2b2d80f01ff752cbd
I20260926 14:58:13.223973 20660 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 14:58:13.226056 20681 tablet.cc:2369] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 14:58:13.332129 20606 tablet.cc:2369] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 14:58:13.380426 20757 tablet.cc:2369] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:58:13.402163 20831 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:58:13.402240 20831 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } } peers { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } } peers { permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } }
I20260926 14:58:13.402452 20831 leader_election.cc:288] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d743b8f143c642c2b2d80f01ff752cbd (127.7.37.253:37137), 0a2c87156a2446bbb8cde25bf31b098f (127.7.37.252:43069)
I20260926 14:58:13.402640 20778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2c87156a2446bbb8cde25bf31b098f" is_pre_election: true
I20260926 14:58:13.402643 20702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d743b8f143c642c2b2d80f01ff752cbd" is_pre_election: true
I20260926 14:58:13.402753 20778 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87b895c02e4a46ee8b1ad0f9f72cfe30 in term 0.
I20260926 14:58:13.402755 20702 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87b895c02e4a46ee8b1ad0f9f72cfe30 in term 0.
I20260926 14:58:13.402901 20603 leader_election.cc:302] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87b895c02e4a46ee8b1ad0f9f72cfe30, d743b8f143c642c2b2d80f01ff752cbd; no voters:
I20260926 14:58:13.403034 20831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:58:13.403090 20831 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:58:13.403105 20831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:13.403688 20831 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } } peers { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } } peers { permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } }
I20260926 14:58:13.403816 20831 leader_election.cc:288] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [CANDIDATE]: Term 1 election: Requested vote from peers d743b8f143c642c2b2d80f01ff752cbd (127.7.37.253:37137), 0a2c87156a2446bbb8cde25bf31b098f (127.7.37.252:43069)
I20260926 14:58:13.403932 20702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d743b8f143c642c2b2d80f01ff752cbd"
I20260926 14:58:13.403959 20778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2c87156a2446bbb8cde25bf31b098f"
I20260926 14:58:13.404018 20778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:13.404013 20702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:13.404767 20702 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87b895c02e4a46ee8b1ad0f9f72cfe30 in term 1.
I20260926 14:58:13.404767 20778 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87b895c02e4a46ee8b1ad0f9f72cfe30 in term 1.
I20260926 14:58:13.404918 20603 leader_election.cc:302] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0a2c87156a2446bbb8cde25bf31b098f, 87b895c02e4a46ee8b1ad0f9f72cfe30, d743b8f143c642c2b2d80f01ff752cbd; no voters:
I20260926 14:58:13.405004 20831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:58:13.405093 20831 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [term 1 LEADER]: Becoming Leader. State: Replica: 87b895c02e4a46ee8b1ad0f9f72cfe30, State: Running, Role: LEADER
I20260926 14:58:13.405160 20831 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } } peers { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } } peers { permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } }
I20260926 14:58:13.405503 20858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87b895c02e4a46ee8b1ad0f9f72cfe30. Latest consensus state: current_term: 1 leader_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } } peers { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } } peers { permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } } }
I20260926 14:58:13.405584 20858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [sys.catalog]: This master's current role is: LEADER
I20260926 14:58:13.405751 20860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:58:13.405895 20860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:58:13.406960 20702 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd [term 1 FOLLOWER]: Refusing update from remote peer 87b895c02e4a46ee8b1ad0f9f72cfe30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:58:13.406962 20778 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f [term 1 FOLLOWER]: Refusing update from remote peer 87b895c02e4a46ee8b1ad0f9f72cfe30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:58:13.407195 20831 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:58:13.407290 20858 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:58:13.408355 20817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87b895c02e4a46ee8b1ad0f9f72cfe30. Latest consensus state: current_term: 1 leader_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } } peers { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } } peers { permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } } }
I20260926 14:58:13.408370 20845 sys_catalog.cc:455] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87b895c02e4a46ee8b1ad0f9f72cfe30. Latest consensus state: current_term: 1 leader_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } } peers { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } } peers { permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } } }
I20260926 14:58:13.408434 20817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f [sys.catalog]: This master's current role is: FOLLOWER
I20260926 14:58:13.408439 20845 sys_catalog.cc:458] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd [sys.catalog]: This master's current role is: FOLLOWER
I20260926 14:58:13.409389 20831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } } peers { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } } peers { permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } } }
I20260926 14:58:13.409394 20845 sys_catalog.cc:455] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } } peers { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } } peers { permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } } }
I20260926 14:58:13.409412 20858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } } peers { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } } peers { permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } } }
I20260926 14:58:13.409457 20845 sys_catalog.cc:458] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd [sys.catalog]: This master's current role is: FOLLOWER
I20260926 14:58:13.409457 20831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [sys.catalog]: This master's current role is: LEADER
I20260926 14:58:13.409477 20858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30 [sys.catalog]: This master's current role is: LEADER
I20260926 14:58:13.409770 20862 mvcc.cc:205] Tried to move back new op lower bound from 7333620504193286144 to 7333620504188063744. Current Snapshot: MvccSnapshot[applied={T|T < 7333620504193286144}]
I20260926 14:58:13.409922 20860 catalog_manager.cc:1347] Generated new cluster ID: 8bf6d5de7c47415d8c1ac36c314ec0bc
I20260926 14:58:13.409958 20860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:58:13.409969 20817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b895c02e4a46ee8b1ad0f9f72cfe30" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39227 } } peers { permanent_uuid: "d743b8f143c642c2b2d80f01ff752cbd" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 37137 } } peers { permanent_uuid: "0a2c87156a2446bbb8cde25bf31b098f" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 43069 } } }
I20260926 14:58:13.410055 20817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f [sys.catalog]: This master's current role is: FOLLOWER
I20260926 14:58:13.437588 20860 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:58:13.438064 20860 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:58:13.442272 20860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87b895c02e4a46ee8b1ad0f9f72cfe30: Generated new TSK 0
I20260926 14:58:13.442423 20860 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:58:14.169821 20828 catalog_manager.cc:1259] Loaded cluster ID: 8bf6d5de7c47415d8c1ac36c314ec0bc
I20260926 14:58:14.169878 20828 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f: loading cluster ID for follower catalog manager: success
I20260926 14:58:14.170620 20828 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f: acquiring CA information for follower catalog manager: success
I20260926 14:58:14.170858 20828 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0a2c87156a2446bbb8cde25bf31b098f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 14:58:14.222504 20856 catalog_manager.cc:1259] Loaded cluster ID: 8bf6d5de7c47415d8c1ac36c314ec0bc
I20260926 14:58:14.222560 20856 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd: loading cluster ID for follower catalog manager: success
I20260926 14:58:14.223294 20856 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd: acquiring CA information for follower catalog manager: success
I20260926 14:58:14.223526 20856 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d743b8f143c642c2b2d80f01ff752cbd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 14:58:14.507453 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 20585
I20260926 14:58:14.511638 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 20660
I20260926 14:58:14.515497 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 20735
2026-09-26T14:58:14Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2838 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:58:14 dist-test-slave-f9xb krb5kdc[20875](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:58:14 dist-test-slave-f9xb krb5kdc[20875](info): set up 2 sockets
Sep 26 14:58:14 dist-test-slave-f9xb krb5kdc[20875](info): commencing operation
krb5kdc: starting...
W20260926 14:58:15.451391 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.918s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:58:15 dist-test-slave-f9xb krb5kdc[20875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:58:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:58:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:58:15.556615 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:39061
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:35683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:39061
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:58:15.631579 20888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:15.631767 20888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:15.631788 20888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:15.632887 20888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:58:15.632925 20888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:15.632937 20888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:15.632951 20888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:58:15.632972 20888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:58:15.634346 20888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:35683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:39061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:39061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:15.634550 20888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:15.634732 20888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:15.638919 20893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:15.638936 20896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:15.638907 20894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:15.639036 20888 server_base.cc:1034] running on GCE node
I20260926 14:58:15.639303 20888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:15.639542 20888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:15.640683 20888 hybrid_clock.cc:648] HybridClock initialized: now 1790434695640670 us; error 28 us; skew 500 ppm
Sep 26 14:58:15 dist-test-slave-f9xb krb5kdc[20875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434695, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:15.642825 20888 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:58:15.643174 20888 webserver.cc:460] Webserver started at http://127.7.37.254:40141/ using document root <none> and password file <none>
I20260926 14:58:15.643347 20888 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:15.643389 20888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:15.643502 20888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:58:15.644447 20888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "6f454cf3ef714b34baad13d7a02c11fd"
format_stamp: "Formatted at 2026-09-26 14:58:15 on dist-test-slave-f9xb"
server_key: "40d47c71ba01be52fcdbc3b9483e0242"
server_key_iv: "2031083adc04b314f7722bf65266e11b"
server_key_version: "encryptionkey@0"
I20260926 14:58:15.644738 20888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "6f454cf3ef714b34baad13d7a02c11fd"
format_stamp: "Formatted at 2026-09-26 14:58:15 on dist-test-slave-f9xb"
server_key: "40d47c71ba01be52fcdbc3b9483e0242"
server_key_iv: "2031083adc04b314f7722bf65266e11b"
server_key_version: "encryptionkey@0"
I20260926 14:58:15.645843 20888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 14:58:15.646426 20903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:15.646557 20888 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:58:15.646628 20888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "6f454cf3ef714b34baad13d7a02c11fd"
format_stamp: "Formatted at 2026-09-26 14:58:15 on dist-test-slave-f9xb"
server_key: "40d47c71ba01be52fcdbc3b9483e0242"
server_key_iv: "2031083adc04b314f7722bf65266e11b"
server_key_version: "encryptionkey@0"
I20260926 14:58:15.646688 20888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:15.662072 20888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:15.663874 20888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:15.663987 20888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:15.667169 20888 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:39061
I20260926 14:58:15.667201 20955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:39061 every 8 connection(s)
I20260926 14:58:15.667452 20888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:58:15.668184 20956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:15.670022 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 20888
I20260926 14:58:15.670116 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:58:15.670179 20956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd: Bootstrap starting.
I20260926 14:58:15.670307 7319 external_mini_cluster.cc:1428] Setting key 6afe565b902b9478d6f1e99362142868
I20260926 14:58:15.670763 20956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:15.671041 20956 log.cc:824] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:15.671567 20956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd: No bootstrap required, opened a new log
I20260926 14:58:15.673207 20956 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f454cf3ef714b34baad13d7a02c11fd" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39061 } }
I20260926 14:58:15.673323 20956 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:15.673352 20956 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f454cf3ef714b34baad13d7a02c11fd, State: Initialized, Role: FOLLOWER
I20260926 14:58:15.673473 20956 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f454cf3ef714b34baad13d7a02c11fd" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39061 } }
I20260926 14:58:15.673543 20956 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:58:15.673573 20956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:58:15.673615 20956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:15.674060 20956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f454cf3ef714b34baad13d7a02c11fd" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39061 } }
I20260926 14:58:15.674146 20956 leader_election.cc:302] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f454cf3ef714b34baad13d7a02c11fd; no voters:
I20260926 14:58:15.674296 20956 leader_election.cc:288] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:58:15.674330 20961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:58:15.674437 20961 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd [term 1 LEADER]: Becoming Leader. State: Replica: 6f454cf3ef714b34baad13d7a02c11fd, State: Running, Role: LEADER
I20260926 14:58:15.674535 20956 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:58:15.674538 20961 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f454cf3ef714b34baad13d7a02c11fd" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39061 } }
I20260926 14:58:15.674969 20962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f454cf3ef714b34baad13d7a02c11fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f454cf3ef714b34baad13d7a02c11fd" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39061 } } }
I20260926 14:58:15.675047 20962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd [sys.catalog]: This master's current role is: LEADER
I20260926 14:58:15.675228 20963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f454cf3ef714b34baad13d7a02c11fd. Latest consensus state: current_term: 1 leader_uuid: "6f454cf3ef714b34baad13d7a02c11fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f454cf3ef714b34baad13d7a02c11fd" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 39061 } } }
I20260926 14:58:15.675329 20963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd [sys.catalog]: This master's current role is: LEADER
W20260926 14:58:15.676183 20977 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 14:58:15.676236 20977 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 26 14:58:15 dist-test-slave-f9xb krb5kdc[20875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:15.676333 20968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:58:15.676528 20968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:58:15.677805 20968 catalog_manager.cc:1347] Generated new cluster ID: 83478159b8ab4ff0948a8d9b0dc11e23
I20260926 14:58:15.677842 20968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:58:15.680246 20959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:15.670689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45266 (local address 127.7.37.254:39061)
0926 14:58:15.670854 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:15.670862 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:15.670890 (+ 28us) server_negotiation.cc:407] Connection header received
0926 14:58:15.671306 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:15.671312 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:15.671355 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:15.671467 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:15.672112 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:15.672945 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:15.673708 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:15.674087 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:15.674173 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:15.676717 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:15.676721 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:15.676721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:15.676731 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:15.678813 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:15.679185 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:15.679187 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:15.679187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:15.679272 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:15.679451 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:15.679451 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:15.679451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:15.679696 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:15.679848 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:15.679952 (+ 104us) server_negotiation.cc:299] Negotiation successful
0926 14:58:15.680067 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
I20260926 14:58:15.680996 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:58:15.692696 20968 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:58:15.693151 20968 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:58:15.696587 20968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f454cf3ef714b34baad13d7a02c11fd: Generated new TSK 0
I20260926 14:58:15.696712 20968 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:58:15.734068 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 20888
2026-09-26T14:58:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1217 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:58:15 dist-test-slave-f9xb krb5kdc[20983](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:58:15 dist-test-slave-f9xb krb5kdc[20983](info): set up 2 sockets
Sep 26 14:58:15 dist-test-slave-f9xb krb5kdc[20983](info): commencing operation
krb5kdc: starting...
W20260926 14:58:16.671367 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.920s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:58:16 dist-test-slave-f9xb krb5kdc[20983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:58:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:58:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:58:16.772696 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36099
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:36099
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:58:16.848532 20996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:16.848706 20996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:16.848734 20996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:16.849814 20996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:58:16.849850 20996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:16.849869 20996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:16.849884 20996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:58:16.849900 20996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:58:16.851287 20996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:36099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:16.851497 20996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:16.851673 20996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:16.855901 21004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:16.855922 21002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:16.855909 21001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:16.855938 20996 server_base.cc:1034] running on GCE node
I20260926 14:58:16.856274 20996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:16.856518 20996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:16.857648 20996 hybrid_clock.cc:648] HybridClock initialized: now 1790434696857633 us; error 28 us; skew 500 ppm
Sep 26 14:58:16 dist-test-slave-f9xb krb5kdc[20983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434696, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:16.859704 20996 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:58:16.860085 20996 webserver.cc:460] Webserver started at http://127.7.37.254:34709/ using document root <none> and password file <none>
I20260926 14:58:16.860271 20996 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:16.860316 20996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:16.860424 20996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:58:16.861307 20996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "d918869b52414516bf5ccaa937ec33b3"
format_stamp: "Formatted at 2026-09-26 14:58:16 on dist-test-slave-f9xb"
server_key: "af7ea4765c5e19eff5566bd087ac2356"
server_key_iv: "ac5083022859dda767ebf91def893dd8"
server_key_version: "encryptionkey@0"
I20260926 14:58:16.861601 20996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "d918869b52414516bf5ccaa937ec33b3"
format_stamp: "Formatted at 2026-09-26 14:58:16 on dist-test-slave-f9xb"
server_key: "af7ea4765c5e19eff5566bd087ac2356"
server_key_iv: "ac5083022859dda767ebf91def893dd8"
server_key_version: "encryptionkey@0"
I20260926 14:58:16.862703 20996 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 14:58:16.863274 21011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:16.863411 20996 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:16.863484 20996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "d918869b52414516bf5ccaa937ec33b3"
format_stamp: "Formatted at 2026-09-26 14:58:16 on dist-test-slave-f9xb"
server_key: "af7ea4765c5e19eff5566bd087ac2356"
server_key_iv: "ac5083022859dda767ebf91def893dd8"
server_key_version: "encryptionkey@0"
I20260926 14:58:16.863539 20996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:16.872277 20996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:16.873783 20996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:16.873903 20996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:16.877355 20996 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:36099
I20260926 14:58:16.877385 21063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:36099 every 8 connection(s)
I20260926 14:58:16.877643 20996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:58:16.878470 21064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:16.880388 21064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3: Bootstrap starting.
I20260926 14:58:16.880834 21064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:16.881044 21064 log.cc:824] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:16.881633 21064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3: No bootstrap required, opened a new log
I20260926 14:58:16.883040 21064 raft_consensus.cc:348] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d918869b52414516bf5ccaa937ec33b3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36099 } }
I20260926 14:58:16.883158 21064 raft_consensus.cc:374] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:16.883178 21064 raft_consensus.cc:729] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d918869b52414516bf5ccaa937ec33b3, State: Initialized, Role: FOLLOWER
I20260926 14:58:16.883273 21064 consensus_queue.cc:260] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d918869b52414516bf5ccaa937ec33b3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36099 } }
I20260926 14:58:16.883350 21064 raft_consensus.cc:388] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:58:16.883378 21064 raft_consensus.cc:482] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:58:16.883407 21064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:16.884110 21064 raft_consensus.cc:504] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d918869b52414516bf5ccaa937ec33b3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36099 } }
I20260926 14:58:16.884191 21064 leader_election.cc:302] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d918869b52414516bf5ccaa937ec33b3; no voters:
I20260926 14:58:16.884301 21064 leader_election.cc:288] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:58:16.884351 21067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:58:16.884496 21064 sys_catalog.cc:564] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:58:16.884501 21067 raft_consensus.cc:686] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 1 LEADER]: Becoming Leader. State: Replica: d918869b52414516bf5ccaa937ec33b3, State: Running, Role: LEADER
I20260926 14:58:16.884668 21067 consensus_queue.cc:237] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d918869b52414516bf5ccaa937ec33b3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36099 } }
I20260926 14:58:16.884997 21069 sys_catalog.cc:455] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d918869b52414516bf5ccaa937ec33b3. Latest consensus state: current_term: 1 leader_uuid: "d918869b52414516bf5ccaa937ec33b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d918869b52414516bf5ccaa937ec33b3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36099 } } }
I20260926 14:58:16.885007 21068 sys_catalog.cc:455] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d918869b52414516bf5ccaa937ec33b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d918869b52414516bf5ccaa937ec33b3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36099 } } }
I20260926 14:58:16.885156 21069 sys_catalog.cc:458] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [sys.catalog]: This master's current role is: LEADER
I20260926 14:58:16.885169 21068 sys_catalog.cc:458] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [sys.catalog]: This master's current role is: LEADER
I20260926 14:58:16.885509 21076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:58:16.885952 21076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:58:16.886247 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 20996
I20260926 14:58:16.886363 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:58:16.886555 7319 external_mini_cluster.cc:1428] Setting key 85548e5c767433c5df7c41faad86097c
I20260926 14:58:16.887305 21076 catalog_manager.cc:1347] Generated new cluster ID: e7aa72c553ac45c28c8292b407daffe5
I20260926 14:58:16.887347 21076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 14:58:16 dist-test-slave-f9xb krb5kdc[20983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:16.897244 21086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:16.886892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53110 (local address 127.7.37.254:36099)
0926 14:58:16.887551 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:16.887558 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:16.887584 (+ 26us) server_negotiation.cc:407] Connection header received
0926 14:58:16.887681 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:16.887687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:16.887735 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:16.887888 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:16.888546 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:16.889923 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:16.890872 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:16.891468 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:16.891571 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:16.894059 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:16.894062 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:16.894063 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:16.894073 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:16.895846 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:16.896198 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:16.896200 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:16.896201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:16.896288 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:16.896461 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:16.896462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:16.896462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:16.896710 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:16.896842 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:16.896931 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 14:58:16.897048 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":64,"threads_started":1}
I20260926 14:58:16.898701 21076 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:58:16.899343 21076 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:58:16.914021 21076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3: Generated new TSK 0
I20260926 14:58:16.914171 21076 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
I20260926 14:58:16.938192 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36099
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:58:17.013330 21090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:17.013502 21090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:17.013525 21090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:17.014578 21090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:17.014608 21090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:17.014632 21090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:58:17.015965 21090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:17.016131 21090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:17.016283 21090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:17.020668 21098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:17.020677 21095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:17.020774 21090 server_base.cc:1034] running on GCE node
W20260926 14:58:17.020704 21096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:17.021056 21090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:17.021278 21090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:17.022411 21090 hybrid_clock.cc:648] HybridClock initialized: now 1790434697022401 us; error 26 us; skew 500 ppm
Sep 26 14:58:17 dist-test-slave-f9xb krb5kdc[20983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434697, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:17.024648 21090 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:58:17.025018 21090 webserver.cc:460] Webserver started at http://127.7.37.193:34023/ using document root <none> and password file <none>
I20260926 14:58:17.025193 21090 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:17.025238 21090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:17.025349 21090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:58:17.026232 21090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "db551780aa8941bb825d07a910af1d7d"
format_stamp: "Formatted at 2026-09-26 14:58:17 on dist-test-slave-f9xb"
server_key: "e0a7f4d49066216a53849fe80818bfa6"
server_key_iv: "ac247eefae177f7acad12b617cde873c"
server_key_version: "encryptionkey@0"
I20260926 14:58:17.026533 21090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "db551780aa8941bb825d07a910af1d7d"
format_stamp: "Formatted at 2026-09-26 14:58:17 on dist-test-slave-f9xb"
server_key: "e0a7f4d49066216a53849fe80818bfa6"
server_key_iv: "ac247eefae177f7acad12b617cde873c"
server_key_version: "encryptionkey@0"
I20260926 14:58:17.027688 21090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:58:17.028373 21105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:17.028512 21090 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:17.028586 21090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "db551780aa8941bb825d07a910af1d7d"
format_stamp: "Formatted at 2026-09-26 14:58:17 on dist-test-slave-f9xb"
server_key: "e0a7f4d49066216a53849fe80818bfa6"
server_key_iv: "ac247eefae177f7acad12b617cde873c"
server_key_version: "encryptionkey@0"
I20260926 14:58:17.028642 21090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:17.063731 21090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:17.065462 21090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:17.065584 21090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:17.065773 21090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:17.066102 21090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:58:17.066136 21090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:17.066164 21090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:58:17.066186 21090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:17.071408 21090 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:33929
I20260926 14:58:17.071440 21218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:33929 every 8 connection(s)
I20260926 14:58:17.071707 21090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:58:17.071892 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 21090
I20260926 14:58:17.072005 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:58:17.072178 7319 external_mini_cluster.cc:1428] Setting key ca8ddefeba4c0b4079aeb5c22232958c
Sep 26 14:58:17 dist-test-slave-f9xb krb5kdc[20983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434697, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.37.194@KRBTEST.COM; defaulting to no policy
I20260926 14:58:17.080652 21086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:17.072445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:56273 (local address 127.7.37.254:36099)
0926 14:58:17.072502 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:17.072505 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:17.072632 (+ 127us) server_negotiation.cc:407] Connection header received
0926 14:58:17.073198 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:17.073198 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:17.073210 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:17.073256 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:17.074353 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.074844 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.075807 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.076120 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.076198 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:17.078022 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:17.078025 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:17.078025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:17.078036 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:17.079430 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:17.079803 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:17.079804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:17.079805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:17.079858 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:17.080054 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:17.080055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:17.080055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:17.080155 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:17.080211 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:17.080466 (+ 255us) server_negotiation.cc:299] Negotiation successful
0926 14:58:17.080561 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 14:58:17.080798 21221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:17.072350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:56273)
0926 14:58:17.072526 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:17.072567 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:17.073105 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:17.073350 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:17.073354 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:17.074009 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:17.074266 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:17.074282 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.074921 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.074922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:17.075698 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:17.075700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.076200 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.076201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:17.076303 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:17.076733 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:17.076740 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:17.077941 (+ 1201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:17.079499 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:17.079504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:17.079506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:17.079706 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:17.079911 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:17.079912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:17.079912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:17.080001 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:17.080208 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:17.080214 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:17.080353 (+ 139us) client_negotiation.cc:769] Sending connection context
0926 14:58:17.080435 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 14:58:17.080609 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260926 14:58:17.081167 21219 heartbeater.cc:347] Connected to a master server at 127.7.37.254:36099
I20260926 14:58:17.081236 21219 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:17.081367 21219 heartbeater.cc:511] Master 127.7.37.254:36099 requested a full tablet report, sending...
I20260926 14:58:17.081812 21028 ts_manager.cc:194] Registered new tserver with Master: db551780aa8941bb825d07a910af1d7d (127.7.37.193:33929)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.194@KRBTEST.COM" created.
I20260926 14:58:17.082485 21028 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:56273
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
I20260926 14:58:17.094954 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36099
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:58:17.169565 21224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:17.169732 21224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:17.169752 21224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:17.170817 21224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:17.170853 21224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:17.170876 21224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:58:17.172223 21224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:17.172396 21224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:17.172539 21224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:17.176853 21230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:17.176868 21232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:17.176939 21224 server_base.cc:1034] running on GCE node
W20260926 14:58:17.176865 21229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:17.177218 21224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:17.177428 21224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:17.178560 21224 hybrid_clock.cc:648] HybridClock initialized: now 1790434697178546 us; error 27 us; skew 500 ppm
Sep 26 14:58:17 dist-test-slave-f9xb krb5kdc[20983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434697, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:17.180729 21224 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:58:17.181108 21224 webserver.cc:460] Webserver started at http://127.7.37.194:32991/ using document root <none> and password file <none>
I20260926 14:58:17.181288 21224 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:17.181331 21224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:17.181438 21224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:58:17.182298 21224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50"
format_stamp: "Formatted at 2026-09-26 14:58:17 on dist-test-slave-f9xb"
server_key: "301342a95ebbcc84d110f65e3b4f9287"
server_key_iv: "f09d8a14d76d91b0d2a5e564545b16c1"
server_key_version: "encryptionkey@0"
I20260926 14:58:17.182587 21224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50"
format_stamp: "Formatted at 2026-09-26 14:58:17 on dist-test-slave-f9xb"
server_key: "301342a95ebbcc84d110f65e3b4f9287"
server_key_iv: "f09d8a14d76d91b0d2a5e564545b16c1"
server_key_version: "encryptionkey@0"
I20260926 14:58:17.183827 21224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 14:58:17.184484 21239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:17.184626 21224 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:58:17.184700 21224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50"
format_stamp: "Formatted at 2026-09-26 14:58:17 on dist-test-slave-f9xb"
server_key: "301342a95ebbcc84d110f65e3b4f9287"
server_key_iv: "f09d8a14d76d91b0d2a5e564545b16c1"
server_key_version: "encryptionkey@0"
I20260926 14:58:17.184763 21224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:17.192489 21224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:17.193980 21224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:17.194092 21224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:17.194252 21224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:17.194499 21224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:58:17.194528 21224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:17.194552 21224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:58:17.194576 21224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:17.199671 21224 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:42267
I20260926 14:58:17.199702 21352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:42267 every 8 connection(s)
I20260926 14:58:17.199956 21224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
Sep 26 14:58:17 dist-test-slave-f9xb krb5kdc[20983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434697, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:17.207984 21086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:17.200479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:52381 (local address 127.7.37.254:36099)
0926 14:58:17.200526 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:17.200528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:17.200692 (+ 164us) server_negotiation.cc:407] Connection header received
0926 14:58:17.201096 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:17.201097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:17.201109 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:17.201158 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:17.201952 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.202441 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.203333 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.203540 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.203594 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:17.205514 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:17.205516 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:17.205516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:17.205523 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:17.206659 (+ 1136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:17.207045 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:17.207046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:17.207047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:17.207106 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:17.207344 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:17.207345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:17.207346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:17.207461 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:17.207521 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:17.207798 (+ 277us) server_negotiation.cc:299] Negotiation successful
0926 14:58:17.207890 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 14:58:17.208093 21355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:17.200466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:52381)
0926 14:58:17.200596 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:17.200626 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:17.201013 (+ 387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:17.201214 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:17.201218 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:17.201706 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:17.201868 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:17.201872 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.202517 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.202518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:17.203254 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:17.203256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.203602 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.203604 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:17.203713 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:17.204135 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:17.204143 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:17.205425 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:17.206749 (+ 1324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:17.206756 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:17.206759 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:17.206972 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:17.207163 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:17.207164 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:17.207165 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:17.207283 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:17.207518 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:17.207524 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:17.207648 (+ 124us) client_negotiation.cc:769] Sending connection context
0926 14:58:17.207736 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 14:58:17.207899 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":72,"threads_started":1}
I20260926 14:58:17.208458 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 21224
I20260926 14:58:17.208519 21353 heartbeater.cc:347] Connected to a master server at 127.7.37.254:36099
I20260926 14:58:17.208557 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:58:17.208602 21353 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:17.208746 7319 external_mini_cluster.cc:1428] Setting key 1a3968837491e6aefb3adc741165b8ad
I20260926 14:58:17.208761 21353 heartbeater.cc:511] Master 127.7.37.254:36099 requested a full tablet report, sending...
I20260926 14:58:17.209148 21027 ts_manager.cc:194] Registered new tserver with Master: fe0eb2dcfa8c4a11b055b11db8a96d50 (127.7.37.194:42267)
I20260926 14:58:17.209575 21027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:52381
WARNING: no policy specified for kudu/127.7.37.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
I20260926 14:58:17.229779 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36099
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:58:17.306950 21358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:17.307116 21358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:17.307137 21358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:17.308252 21358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:17.308290 21358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:17.308314 21358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:58:17.309675 21358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:17.309865 21358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:17.310071 21358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:17.314558 21363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:17.314580 21366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:17.314584 21364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:17.314754 21358 server_base.cc:1034] running on GCE node
I20260926 14:58:17.314955 21358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:17.315160 21358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:17.316308 21358 hybrid_clock.cc:648] HybridClock initialized: now 1790434697316291 us; error 27 us; skew 500 ppm
Sep 26 14:58:17 dist-test-slave-f9xb krb5kdc[20983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434697, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:17.318503 21358 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:58:17.318898 21358 webserver.cc:460] Webserver started at http://127.7.37.195:45035/ using document root <none> and password file <none>
I20260926 14:58:17.319072 21358 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:17.319109 21358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:17.319193 21358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:58:17.320111 21358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "cfbb02f44e964cc3870e342766ab2911"
format_stamp: "Formatted at 2026-09-26 14:58:17 on dist-test-slave-f9xb"
server_key: "638ba429fedbb853ac039097d2f5e98d"
server_key_iv: "76514744c2873f38823412d51f258432"
server_key_version: "encryptionkey@0"
I20260926 14:58:17.320394 21358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "cfbb02f44e964cc3870e342766ab2911"
format_stamp: "Formatted at 2026-09-26 14:58:17 on dist-test-slave-f9xb"
server_key: "638ba429fedbb853ac039097d2f5e98d"
server_key_iv: "76514744c2873f38823412d51f258432"
server_key_version: "encryptionkey@0"
I20260926 14:58:17.321487 21358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:58:17.322098 21373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:17.322297 21358 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:58:17.322373 21358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "cfbb02f44e964cc3870e342766ab2911"
format_stamp: "Formatted at 2026-09-26 14:58:17 on dist-test-slave-f9xb"
server_key: "638ba429fedbb853ac039097d2f5e98d"
server_key_iv: "76514744c2873f38823412d51f258432"
server_key_version: "encryptionkey@0"
I20260926 14:58:17.322440 21358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:17.337316 21358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:17.338825 21358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:17.338941 21358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:17.339109 21358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:17.339355 21358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:58:17.339385 21358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:17.339411 21358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:58:17.339434 21358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:17.344610 21358 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:38377
I20260926 14:58:17.344643 21486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:38377 every 8 connection(s)
I20260926 14:58:17.344890 21358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
Sep 26 14:58:17 dist-test-slave-f9xb krb5kdc[20983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434697, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:17.352820 21086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:17.345489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:54687 (local address 127.7.37.254:36099)
0926 14:58:17.345540 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:17.345542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:17.345720 (+ 178us) server_negotiation.cc:407] Connection header received
0926 14:58:17.346114 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:17.346115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:17.346125 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:17.346173 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:17.346977 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.347554 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.348561 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.348745 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.348797 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:17.350664 (+ 1867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:17.350665 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:17.350666 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:17.350672 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:17.351698 (+ 1026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:17.352084 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:17.352085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:17.352086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:17.352141 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:17.352373 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:17.352373 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:17.352374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:17.352468 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:17.352514 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:17.352704 (+ 190us) server_negotiation.cc:299] Negotiation successful
0926 14:58:17.352761 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 14:58:17.352941 21489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:17.345463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:54687)
0926 14:58:17.345616 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:17.345646 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:17.346040 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:17.346247 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:17.346250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:17.346745 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:17.346911 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:17.346915 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.347638 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.347639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:17.348481 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:17.348484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.348805 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.348806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:17.348899 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:17.349293 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:17.349300 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:17.350572 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:17.351798 (+ 1226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:17.351804 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:17.351806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:17.352002 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:17.352223 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:17.352223 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:17.352224 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:17.352322 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:17.352512 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:17.352515 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:17.352613 (+ 98us) client_negotiation.cc:769] Sending connection context
0926 14:58:17.352683 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 14:58:17.352788 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260926 14:58:17.353317 21487 heartbeater.cc:347] Connected to a master server at 127.7.37.254:36099
I20260926 14:58:17.353404 21487 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:17.353525 21487 heartbeater.cc:511] Master 127.7.37.254:36099 requested a full tablet report, sending...
I20260926 14:58:17.353705 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 21358
I20260926 14:58:17.353797 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:58:17.353852 21027 ts_manager.cc:194] Registered new tserver with Master: cfbb02f44e964cc3870e342766ab2911 (127.7.37.195:38377)
I20260926 14:58:17.353978 7319 external_mini_cluster.cc:1428] Setting key 49a18e03d4f192798629babdf8dfc3a7
I20260926 14:58:17.354247 21027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:54687
I20260926 14:58:17.354768 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 14:58:17.361136 21086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:17.356002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53116 (local address 127.7.37.254:36099)
0926 14:58:17.356031 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:17.356033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:17.356150 (+ 117us) server_negotiation.cc:407] Connection header received
0926 14:58:17.356204 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:17.356205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:17.356214 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:17.356258 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:17.356760 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.357208 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.357891 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.358085 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.358133 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:17.359096 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:17.359097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:17.359098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:17.359103 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:17.360201 (+ 1098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:17.360483 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:17.360484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:17.360484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:17.360534 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:17.360684 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:17.360685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:17.360685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:17.360806 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:17.360902 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:17.360991 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 14:58:17.361058 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260926 14:58:17.362555 21027 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:58:17.364114 21027 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:58:17.369685 21496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:17.365897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:42267 (local address 127.0.0.1:43408)
0926 14:58:17.366000 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:17.366034 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:17.366092 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:17.366397 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:17.366399 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:17.366401 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:17.366462 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:17.366464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.367825 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.367826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:17.368574 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:17.368575 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.369417 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.369418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:17.369516 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:17.369530 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 14:58:17.369575 (+ 45us) client_negotiation.cc:240] Negotiation successful
0926 14:58:17.369621 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":41,"threads_started":1}
I20260926 14:58:17.370012 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:17.365904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:33929 (local address 127.0.0.1:51504)
0926 14:58:17.366235 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:17.366245 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:17.366303 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:17.366481 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:17.366481 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:17.366483 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:17.366523 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:17.366524 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.367782 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.367783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:17.369063 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:17.369065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.369692 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.369693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:17.369844 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:17.369847 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:58:17.369901 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 14:58:17.369938 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":4,"thread_start_us":57,"threads_started":1}
I20260926 14:58:17.370016 21499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:17.366044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43408 (local address 127.7.37.194:42267)
0926 14:58:17.366216 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:17.366219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:17.366229 (+ 10us) server_negotiation.cc:407] Connection header received
0926 14:58:17.366267 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:17.366269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:17.366284 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:17.366369 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:17.366542 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.367727 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.368663 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.369316 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.369784 (+ 468us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:17.369862 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:17.369889 (+ 27us) server_negotiation.cc:299] Negotiation successful
0926 14:58:17.369927 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":77,"threads_started":1}
I20260926 14:58:17.370044 21500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:17.366118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51504 (local address 127.7.37.193:33929)
0926 14:58:17.366256 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:17.366260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:17.366291 (+ 31us) server_negotiation.cc:407] Connection header received
0926 14:58:17.366378 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:17.366380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:17.366395 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:17.366475 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:17.366582 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.367669 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.369130 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.369620 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.369680 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:17.369734 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:17.369918 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 14:58:17.369962 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":76,"threads_started":1}
I20260926 14:58:17.370393 21498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:17.366067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58494 (local address 127.7.37.195:38377)
0926 14:58:17.366188 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:17.366192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:17.366540 (+ 348us) server_negotiation.cc:407] Connection header received
0926 14:58:17.366635 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:17.366637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:17.366650 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:17.366705 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:17.366820 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.367565 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.369013 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.369663 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.369720 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:17.370201 (+ 481us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:17.370221 (+ 20us) server_negotiation.cc:299] Negotiation successful
0926 14:58:17.370249 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":55,"threads_started":1}
I20260926 14:58:17.370527 21495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:17.365904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:38377 (local address 127.0.0.1:58494)
0926 14:58:17.366458 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:17.366467 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:17.366563 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:17.366711 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:17.366712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:17.366713 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:17.366755 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:17.366756 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.367665 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.367666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:17.368938 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:17.368941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.369893 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.369893 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:17.369960 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:17.369964 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:58:17.370015 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 14:58:17.370467 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"mutex_wait_us":36,"thread_start_us":66,"threads_started":1}
I20260926 14:58:17.370645 21287 tablet_service.cc:1467] Processing CreateTablet for tablet 9ad6bf1809fa4999976fc6913115f755 (DEFAULT_TABLE table=test-table [id=f1cf0a8c648c4fdb81b1ebc4154e6c09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:58:17.370793 21153 tablet_service.cc:1467] Processing CreateTablet for tablet 9ad6bf1809fa4999976fc6913115f755 (DEFAULT_TABLE table=test-table [id=f1cf0a8c648c4fdb81b1ebc4154e6c09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:58:17.370958 21287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ad6bf1809fa4999976fc6913115f755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:17.371090 21153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ad6bf1809fa4999976fc6913115f755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:17.371104 21421 tablet_service.cc:1467] Processing CreateTablet for tablet 9ad6bf1809fa4999976fc6913115f755 (DEFAULT_TABLE table=test-table [id=f1cf0a8c648c4fdb81b1ebc4154e6c09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:58:17.371348 21421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ad6bf1809fa4999976fc6913115f755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:17.374119 21502 tablet_bootstrap.cc:492] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d: Bootstrap starting.
I20260926 14:58:17.374200 21501 tablet_bootstrap.cc:492] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50: Bootstrap starting.
I20260926 14:58:17.374707 21502 tablet_bootstrap.cc:654] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:17.374737 21501 tablet_bootstrap.cc:654] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:17.375137 21502 log.cc:824] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:17.375155 21501 log.cc:824] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:17.375428 21503 tablet_bootstrap.cc:492] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911: Bootstrap starting.
I20260926 14:58:17.375830 21502 tablet_bootstrap.cc:492] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d: No bootstrap required, opened a new log
I20260926 14:58:17.375830 21501 tablet_bootstrap.cc:492] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50: No bootstrap required, opened a new log
I20260926 14:58:17.375900 21501 ts_tablet_manager.cc:1397] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:58:17.375896 21502 ts_tablet_manager.cc:1397] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:58:17.375957 21503 tablet_bootstrap.cc:654] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:17.376381 21503 log.cc:824] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:17.376928 21503 tablet_bootstrap.cc:492] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911: No bootstrap required, opened a new log
I20260926 14:58:17.376982 21503 ts_tablet_manager.cc:1397] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:58:17.377697 21502 raft_consensus.cc:348] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } }
I20260926 14:58:17.377697 21501 raft_consensus.cc:348] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } }
I20260926 14:58:17.377863 21502 raft_consensus.cc:374] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:17.377863 21501 raft_consensus.cc:374] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:17.377898 21502 raft_consensus.cc:729] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db551780aa8941bb825d07a910af1d7d, State: Initialized, Role: FOLLOWER
I20260926 14:58:17.377907 21501 raft_consensus.cc:729] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe0eb2dcfa8c4a11b055b11db8a96d50, State: Initialized, Role: FOLLOWER
I20260926 14:58:17.378010 21501 consensus_queue.cc:260] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } }
I20260926 14:58:17.378021 21502 consensus_queue.cc:260] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } }
I20260926 14:58:17.378264 21501 ts_tablet_manager.cc:1428] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:58:17.378264 21502 ts_tablet_manager.cc:1428] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:58:17.378371 21353 heartbeater.cc:503] Master 127.7.37.254:36099 was elected leader, sending a full tablet report...
I20260926 14:58:17.378371 21219 heartbeater.cc:503] Master 127.7.37.254:36099 was elected leader, sending a full tablet report...
I20260926 14:58:17.378647 21503 raft_consensus.cc:348] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } }
I20260926 14:58:17.378798 21503 raft_consensus.cc:374] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:17.378836 21503 raft_consensus.cc:729] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfbb02f44e964cc3870e342766ab2911, State: Initialized, Role: FOLLOWER
I20260926 14:58:17.378937 21503 consensus_queue.cc:260] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } }
I20260926 14:58:17.379191 21487 heartbeater.cc:503] Master 127.7.37.254:36099 was elected leader, sending a full tablet report...
I20260926 14:58:17.379181 21503 ts_tablet_manager.cc:1428] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260926 14:58:17.450402 21354 tablet.cc:2369] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 14:58:17.572350 21220 tablet.cc:2369] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 14:58:17.595369 21488 tablet.cc:2369] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:58:17.664937 21507 raft_consensus.cc:482] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:58:17.665038 21507 raft_consensus.cc:504] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } }
I20260926 14:58:17.665371 21507 leader_election.cc:288] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db551780aa8941bb825d07a910af1d7d (127.7.37.193:33929), cfbb02f44e964cc3870e342766ab2911 (127.7.37.195:38377)
I20260926 14:58:17.668038 21500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:17.665648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:54831 (local address 127.7.37.193:33929)
0926 14:58:17.665697 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:17.665698 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:17.665706 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:58:17.665729 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:17.665730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:17.665741 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:17.665787 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:17.665949 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.666414 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.667354 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.667710 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.667804 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:17.667813 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:17.667943 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 14:58:17.667978 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 14:58:17.668038 21355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:17.665538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:33929 (local address 127.7.37.194:54831)
0926 14:58:17.665581 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:17.665593 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:17.665641 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:17.665812 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:17.665813 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:17.665815 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:17.665890 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:17.665891 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.666495 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.666496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:17.667276 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:17.667279 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.667804 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.667805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:17.667882 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:17.667885 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:58:17.667930 (+ 45us) client_negotiation.cc:240] Negotiation successful
0926 14:58:17.667967 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260926 14:58:17.668272 21498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:17.665648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:36191 (local address 127.7.37.195:38377)
0926 14:58:17.665707 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:17.665709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:17.665806 (+ 97us) server_negotiation.cc:407] Connection header received
0926 14:58:17.665863 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:17.665864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:17.665874 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:17.665926 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:17.666067 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.666570 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.667515 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.667919 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.667988 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:17.667996 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:17.668161 (+ 165us) server_negotiation.cc:299] Negotiation successful
0926 14:58:17.668205 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 14:58:17.668306 21173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad6bf1809fa4999976fc6913115f755" candidate_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db551780aa8941bb825d07a910af1d7d" is_pre_election: true
I20260926 14:58:17.668447 21173 raft_consensus.cc:2454] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe0eb2dcfa8c4a11b055b11db8a96d50 in term 0.
I20260926 14:58:17.668478 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:17.665538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:38377 (local address 127.7.37.194:36191)
0926 14:58:17.665716 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:17.665725 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:17.665795 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:17.665935 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:17.665935 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:17.665937 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:17.666008 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:17.666009 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.666657 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.666658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:17.667426 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:17.667428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.667993 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.667994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:17.668062 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:17.668065 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:58:17.668355 (+ 290us) client_negotiation.cc:240] Negotiation successful
0926 14:58:17.668398 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":14,"thread_start_us":96,"threads_started":1}
I20260926 14:58:17.668658 21240 leader_election.cc:302] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db551780aa8941bb825d07a910af1d7d, fe0eb2dcfa8c4a11b055b11db8a96d50; no voters:
I20260926 14:58:17.668761 21507 raft_consensus.cc:2781] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:58:17.668812 21507 raft_consensus.cc:482] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:58:17.668833 21507 raft_consensus.cc:3037] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:17.668792 21441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad6bf1809fa4999976fc6913115f755" candidate_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfbb02f44e964cc3870e342766ab2911" is_pre_election: true
I20260926 14:58:17.668937 21441 raft_consensus.cc:2454] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe0eb2dcfa8c4a11b055b11db8a96d50 in term 0.
I20260926 14:58:17.669504 21507 raft_consensus.cc:504] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } }
I20260926 14:58:17.669615 21507 leader_election.cc:288] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [CANDIDATE]: Term 1 election: Requested vote from peers db551780aa8941bb825d07a910af1d7d (127.7.37.193:33929), cfbb02f44e964cc3870e342766ab2911 (127.7.37.195:38377)
I20260926 14:58:17.669796 21441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad6bf1809fa4999976fc6913115f755" candidate_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfbb02f44e964cc3870e342766ab2911"
I20260926 14:58:17.669786 21173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad6bf1809fa4999976fc6913115f755" candidate_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db551780aa8941bb825d07a910af1d7d"
I20260926 14:58:17.669864 21173 raft_consensus.cc:3037] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:17.669860 21441 raft_consensus.cc:3037] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:17.670490 21441 raft_consensus.cc:2454] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe0eb2dcfa8c4a11b055b11db8a96d50 in term 1.
I20260926 14:58:17.670668 21243 leader_election.cc:302] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cfbb02f44e964cc3870e342766ab2911, fe0eb2dcfa8c4a11b055b11db8a96d50; no voters:
I20260926 14:58:17.670662 21173 raft_consensus.cc:2454] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe0eb2dcfa8c4a11b055b11db8a96d50 in term 1.
I20260926 14:58:17.670825 21507 raft_consensus.cc:2781] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:58:17.670938 21507 raft_consensus.cc:686] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 1 LEADER]: Becoming Leader. State: Replica: fe0eb2dcfa8c4a11b055b11db8a96d50, State: Running, Role: LEADER
I20260926 14:58:17.671023 21507 consensus_queue.cc:237] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } }
I20260926 14:58:17.671717 21027 catalog_manager.cc:5495] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to fe0eb2dcfa8c4a11b055b11db8a96d50 (127.7.37.194). New cstate: current_term: 1 leader_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } health_report { overall_health: UNKNOWN } } }
I20260926 14:58:17.718286 21499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:17.716031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43412 (local address 127.7.37.194:42267)
0926 14:58:17.716076 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:17.716078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:17.716087 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:58:17.716151 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:17.716152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:17.716170 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:17.716237 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:58:17.716393 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.716833 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.717559 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:17.717763 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:17.717822 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:17.717981 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:58:17.718074 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:58:17.718148 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:17.718189 (+ 41us) server_negotiation.cc:299] Negotiation successful
0926 14:58:17.718207 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 14:58:17.719668 21441 raft_consensus.cc:1261] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911 [term 1 FOLLOWER]: Refusing update from remote peer fe0eb2dcfa8c4a11b055b11db8a96d50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:58:17.719672 21173 raft_consensus.cc:1261] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Refusing update from remote peer fe0eb2dcfa8c4a11b055b11db8a96d50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:58:17.719985 21507 consensus_queue.cc:1035] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:58:17.720117 21511 consensus_queue.cc:1035] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:58:17.722877 21516 mvcc.cc:205] Tried to move back new op lower bound from 7333620521857630208 to 7333620521660813312. Current Snapshot: MvccSnapshot[applied={T|T < 7333620521857630208}]
I20260926 14:58:17.724241 21028 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:58:17.724332 21028 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:58:17.725023 21028 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3: Sending DeleteTablet for 3 replicas of tablet 9ad6bf1809fa4999976fc6913115f755
I20260926 14:58:17.725323 21153 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ad6bf1809fa4999976fc6913115f755 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:58:17 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:51504
I20260926 14:58:17.725351 21421 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ad6bf1809fa4999976fc6913115f755 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:58:17 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:58494
I20260926 14:58:17.725416 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 21090
I20260926 14:58:17.726013 21524 tablet_replica.cc:330] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911: stopping tablet replica
I20260926 14:58:17.726109 21524 raft_consensus.cc:2229] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:58:17.726198 21524 raft_consensus.cc:2258] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:58:17.726578 21524 ts_tablet_manager.cc:1905] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:58:17.728374 21287 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ad6bf1809fa4999976fc6913115f755 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:58:17 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:43408
I20260926 14:58:17.728292 21524 ts_tablet_manager.cc:1918] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:58:17.728617 21524 log.cc:1196] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/9ad6bf1809fa4999976fc6913115f755
I20260926 14:58:17.728624 21525 tablet_replica.cc:330] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50: stopping tablet replica
I20260926 14:58:17.728891 21525 raft_consensus.cc:2229] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:58:17.728919 21524 ts_tablet_manager.cc:1939] T 9ad6bf1809fa4999976fc6913115f755 P cfbb02f44e964cc3870e342766ab2911: Deleting consensus metadata
I20260926 14:58:17.729040 21525 raft_consensus.cc:2258] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:58:17.729349 21015 catalog_manager.cc:4841] TS cfbb02f44e964cc3870e342766ab2911 (127.7.37.195:38377): tablet 9ad6bf1809fa4999976fc6913115f755 (table test-table [id=f1cf0a8c648c4fdb81b1ebc4154e6c09]) successfully deleted
I20260926 14:58:17.729532 21525 ts_tablet_manager.cc:1905] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:58:17.730715 21525 ts_tablet_manager.cc:1918] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:58:17.730783 21525 log.cc:1196] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/9ad6bf1809fa4999976fc6913115f755
I20260926 14:58:17.731000 21525 ts_tablet_manager.cc:1939] T 9ad6bf1809fa4999976fc6913115f755 P fe0eb2dcfa8c4a11b055b11db8a96d50: Deleting consensus metadata
I20260926 14:58:17.731334 21013 catalog_manager.cc:4841] TS fe0eb2dcfa8c4a11b055b11db8a96d50 (127.7.37.194:42267): tablet 9ad6bf1809fa4999976fc6913115f755 (table test-table [id=f1cf0a8c648c4fdb81b1ebc4154e6c09]) successfully deleted
W20260926 14:58:17.731766 21012 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.37.193:33929 (error 108)
W20260926 14:58:17.732147 21495 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:58:17.732045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:33929 (local address 127.0.0.1:51518)
0926 14:58:17.732086 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:17.732118 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.193:33929: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":28}
W20260926 14:58:17.732254 21012 catalog_manager.cc:4584] TS db551780aa8941bb825d07a910af1d7d (127.7.37.193:33929): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9ad6bf1809fa4999976fc6913115f755: Network error: Client connection negotiation failed: client connection to 127.7.37.193:33929: connect: Connection reset by peer (error 104)
I20260926 14:58:17.732554 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 21224
I20260926 14:58:17.737215 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 21358
I20260926 14:58:17.742230 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 20996
I20260926 14:58:17.747218 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36099
--webserver_interface=127.7.37.254
--webserver_port=34709
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:36099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:58:17.822484 21527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:17.822646 21527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:17.822666 21527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:17.823698 21527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:58:17.823730 21527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:17.823765 21527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:17.823788 21527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:58:17.823813 21527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:58:17.825152 21527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:36099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=34709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:17.825314 21527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:17.825456 21527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:17.832584 21536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:17.832598 21533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:17.832597 21534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:17.832752 21527 server_base.cc:1034] running on GCE node
I20260926 14:58:17.832962 21527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:17.833232 21527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:17.834358 21527 hybrid_clock.cc:648] HybridClock initialized: now 1790434697834346 us; error 34 us; skew 500 ppm
Sep 26 14:58:17 dist-test-slave-f9xb krb5kdc[20983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434697, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:17.836622 21527 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:58:17.836999 21527 webserver.cc:460] Webserver started at http://127.7.37.254:34709/ using document root <none> and password file <none>
I20260926 14:58:17.837193 21527 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:17.837242 21527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:17.838443 21527 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:58:17.838966 21543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:17.839116 21527 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:58:17.839200 21527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "d918869b52414516bf5ccaa937ec33b3"
format_stamp: "Formatted at 2026-09-26 14:58:16 on dist-test-slave-f9xb"
server_key: "af7ea4765c5e19eff5566bd087ac2356"
server_key_iv: "ac5083022859dda767ebf91def893dd8"
server_key_version: "encryptionkey@0"
I20260926 14:58:17.839521 21527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:17.894984 21527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:17.896628 21527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:17.896723 21527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:17.900117 21527 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:36099
I20260926 14:58:17.900144 21595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:36099 every 8 connection(s)
I20260926 14:58:17.900463 21527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:58:17.901437 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 21527
I20260926 14:58:17.901722 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:33929
--local_ip_for_outbound_sockets=127.7.37.193
--tserver_master_addrs=127.7.37.254:36099
--webserver_port=34023
--webserver_interface=127.7.37.193
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 14:58:17.902056 21596 sys_catalog.cc:263] Verifying existing consensus state
I20260926 14:58:17.902824 21596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3: Bootstrap starting.
I20260926 14:58:17.910435 21596 log.cc:824] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:17.912135 21596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 14:58:17.912364 21596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3: Bootstrap complete.
I20260926 14:58:17.914261 21596 raft_consensus.cc:348] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d918869b52414516bf5ccaa937ec33b3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36099 } }
I20260926 14:58:17.914460 21596 raft_consensus.cc:729] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d918869b52414516bf5ccaa937ec33b3, State: Initialized, Role: FOLLOWER
I20260926 14:58:17.914587 21596 consensus_queue.cc:260] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d918869b52414516bf5ccaa937ec33b3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36099 } }
I20260926 14:58:17.914642 21596 raft_consensus.cc:388] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:58:17.914701 21596 raft_consensus.cc:482] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:58:17.914746 21596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 1 FOLLOWER]: Advancing to term 2
I20260926 14:58:17.915642 21596 raft_consensus.cc:504] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d918869b52414516bf5ccaa937ec33b3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36099 } }
I20260926 14:58:17.915719 21596 leader_election.cc:302] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d918869b52414516bf5ccaa937ec33b3; no voters:
I20260926 14:58:17.915885 21596 leader_election.cc:288] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 14:58:17.915925 21600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 14:58:17.916028 21600 raft_consensus.cc:686] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 2 LEADER]: Becoming Leader. State: Replica: d918869b52414516bf5ccaa937ec33b3, State: Running, Role: LEADER
I20260926 14:58:17.916110 21596 sys_catalog.cc:564] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:58:17.916116 21600 consensus_queue.cc:237] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d918869b52414516bf5ccaa937ec33b3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36099 } }
I20260926 14:58:17.916532 21601 sys_catalog.cc:455] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d918869b52414516bf5ccaa937ec33b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d918869b52414516bf5ccaa937ec33b3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36099 } } }
I20260926 14:58:17.916551 21602 sys_catalog.cc:455] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d918869b52414516bf5ccaa937ec33b3. Latest consensus state: current_term: 2 leader_uuid: "d918869b52414516bf5ccaa937ec33b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d918869b52414516bf5ccaa937ec33b3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36099 } } }
I20260926 14:58:17.916599 21601 sys_catalog.cc:458] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [sys.catalog]: This master's current role is: LEADER
I20260926 14:58:17.916626 21602 sys_catalog.cc:458] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [sys.catalog]: This master's current role is: LEADER
I20260926 14:58:17.917155 21608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:58:17.917843 21608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:58:17.917937 21608 catalog_manager.cc:1259] Loaded cluster ID: e7aa72c553ac45c28c8292b407daffe5
I20260926 14:58:17.917968 21608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:58:17.918170 21608 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260926 14:58:17.922771 21608 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260926 14:58:17.924921 21608 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:58:17.925103 21608 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3: Loaded TSK: 0
I20260926 14:58:17.925315 21608 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 14:58:17.978404 21598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:17.978571 21598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:17.978601 21598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:17.979679 21598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:17.979717 21598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:17.979770 21598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:58:17.981154 21598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:33929
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=34023
--tserver_master_addrs=127.7.37.254:36099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:17.981359 21598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:17.981536 21598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:17.985919 21624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:17.985908 21623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:17.986153 21626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:17.986183 21598 server_base.cc:1034] running on GCE node
I20260926 14:58:17.986403 21598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:17.986649 21598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:17.987841 21598 hybrid_clock.cc:648] HybridClock initialized: now 1790434697987821 us; error 30 us; skew 500 ppm
Sep 26 14:58:17 dist-test-slave-f9xb krb5kdc[20983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434697, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:17.990772 21598 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:58:17.991278 21598 webserver.cc:460] Webserver started at http://127.7.37.193:34023/ using document root <none> and password file <none>
I20260926 14:58:17.991534 21598 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:17.991617 21598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:17.993258 21598 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:58:17.993956 21633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:17.994112 21598 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:58:17.994208 21598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "db551780aa8941bb825d07a910af1d7d"
format_stamp: "Formatted at 2026-09-26 14:58:17 on dist-test-slave-f9xb"
server_key: "e0a7f4d49066216a53849fe80818bfa6"
server_key_iv: "ac247eefae177f7acad12b617cde873c"
server_key_version: "encryptionkey@0"
I20260926 14:58:17.994576 21598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:18.039036 21598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:18.041255 21598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:18.041404 21598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:18.041639 21598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:18.042110 21640 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 14:58:18.043517 21598 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 14:58:18.043574 21598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 14:58:18.043605 21598 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 14:58:18.044533 21598 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 14:58:18.044574 21598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 14:58:18.044620 21640 tablet_bootstrap.cc:492] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d: Bootstrap starting.
I20260926 14:58:18.050865 21598 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:33929
I20260926 14:58:18.051263 21598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:58:18.051793 21747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:33929 every 8 connection(s)
I20260926 14:58:18.054033 21640 log.cc:824] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:18.056015 21640 tablet_bootstrap.cc:492] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 14:58:18.056308 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 21598
I20260926 14:58:18.056540 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:42267
--local_ip_for_outbound_sockets=127.7.37.194
--tserver_master_addrs=127.7.37.254:36099
--webserver_port=32991
--webserver_interface=127.7.37.194
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 14:58:18.057694 21640 tablet_bootstrap.cc:492] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d: Bootstrap complete.
I20260926 14:58:18.058643 21640 ts_tablet_manager.cc:1397] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.005s
Sep 26 14:58:18 dist-test-slave-f9xb krb5kdc[20983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434697, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:18.059767 21640 raft_consensus.cc:348] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } }
I20260926 14:58:18.060003 21640 raft_consensus.cc:729] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db551780aa8941bb825d07a910af1d7d, State: Initialized, Role: FOLLOWER
I20260926 14:58:18.060153 21640 consensus_queue.cc:260] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } }
I20260926 14:58:18.060415 21640 ts_tablet_manager.cc:1428] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:58:18.063588 21750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.052005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:34881 (local address 127.7.37.254:36099)
0926 14:58:18.054114 (+ 2109us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:18.054121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:18.054350 (+ 229us) server_negotiation.cc:407] Connection header received
0926 14:58:18.054948 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:18.054953 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:18.054993 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:18.055124 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:18.055731 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.056734 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.057669 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.058098 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.058197 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:18.060539 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:18.060543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:18.060543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:18.060555 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:18.062024 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:18.062479 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:18.062481 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:18.062481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:18.062559 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:18.062812 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:18.062812 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:18.062813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:18.063002 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:18.063056 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:18.063367 (+ 311us) server_negotiation.cc:299] Negotiation successful
0926 14:58:18.063461 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2074,"thread_start_us":67,"threads_started":1}
I20260926 14:58:18.063668 21751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.051978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:34881)
0926 14:58:18.054195 (+ 2217us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:18.054242 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:18.054753 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:18.055167 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:18.055171 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:18.055434 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:18.055613 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:18.055617 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.056812 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.056813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:18.057591 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:18.057593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.058174 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.058176 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:18.058312 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:18.058790 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:18.058799 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:18.060428 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:18.062101 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:18.062109 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:18.062111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:18.062403 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:18.062615 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:18.062615 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:18.062616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:18.062747 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:18.063058 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:18.063063 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:18.063215 (+ 152us) client_negotiation.cc:769] Sending connection context
0926 14:58:18.063323 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 14:58:18.063482 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2174,"thread_start_us":225,"threads_started":1}
I20260926 14:58:18.064271 21748 heartbeater.cc:347] Connected to a master server at 127.7.37.254:36099
I20260926 14:58:18.064360 21748 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:18.064527 21748 heartbeater.cc:511] Master 127.7.37.254:36099 requested a full tablet report, sending...
I20260926 14:58:18.065182 21560 ts_manager.cc:194] Registered new tserver with Master: db551780aa8941bb825d07a910af1d7d (127.7.37.193:33929)
I20260926 14:58:18.066061 21560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:34881
I20260926 14:58:18.066941 21748 heartbeater.cc:503] Master 127.7.37.254:36099 was elected leader, sending a full tablet report...
Sep 26 14:58:18 dist-test-slave-f9xb krb5kdc[20983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434697, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for kudu/127.7.37.193@KRBTEST.COM
I20260926 14:58:18.072947 21755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.065717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:33929 (local address 127.0.0.1:51532)
0926 14:58:18.066293 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:18.066321 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:18.066383 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:18.066605 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:18.066609 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:18.066838 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:18.066876 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:18.066878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.067799 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.067800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:18.068342 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:18.068343 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.068758 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.068759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:18.068831 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:18.069207 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:18.069211 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:18.070516 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:18.071986 (+ 1470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:18.071992 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:18.071994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:18.072162 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:18.072335 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:18.072336 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:18.072336 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:18.072399 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:18.072629 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:18.072634 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:18.072679 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 14:58:18.072749 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 14:58:18.072852 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":84,"threads_started":1}
I20260926 14:58:18.072957 21756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.065777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51532 (local address 127.7.37.193:33929)
0926 14:58:18.066491 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:18.066494 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:18.066500 (+ 6us) server_negotiation.cc:407] Connection header received
0926 14:58:18.066522 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:18.066523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:18.066536 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:18.066590 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:18.066947 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.067707 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.068420 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.068696 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.068751 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:18.070588 (+ 1837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:18.070595 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:18.070595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:18.070601 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:18.071921 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:18.072230 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:18.072232 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:18.072233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:18.072284 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:18.072446 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:18.072447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:18.072447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:18.072572 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:18.072637 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:18.072789 (+ 152us) server_negotiation.cc:299] Negotiation successful
0926 14:58:18.072878 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":68,"threads_started":1}
I20260926 14:58:18.073272 21667 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ad6bf1809fa4999976fc6913115f755 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:58:17 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:51532
I20260926 14:58:18.073678 21757 tablet_replica.cc:330] stopping tablet replica
I20260926 14:58:18.073782 21757 raft_consensus.cc:2229] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:58:18.073841 21757 raft_consensus.cc:2258] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:58:18.074097 21757 ts_tablet_manager.cc:1905] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:58:18.076093 21757 ts_tablet_manager.cc:1918] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:58:18.076161 21757 log.cc:1196] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/wal/wals/9ad6bf1809fa4999976fc6913115f755
I20260926 14:58:18.076368 21757 ts_tablet_manager.cc:1939] T 9ad6bf1809fa4999976fc6913115f755 P db551780aa8941bb825d07a910af1d7d: Deleting consensus metadata
I20260926 14:58:18.076790 21544 catalog_manager.cc:4841] TS db551780aa8941bb825d07a910af1d7d (127.7.37.193:33929): tablet 9ad6bf1809fa4999976fc6913115f755 (table test-table [id=f1cf0a8c648c4fdb81b1ebc4154e6c09]) successfully deleted
W20260926 14:58:18.137061 21753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:18.137234 21753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:18.137260 21753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:18.138375 21753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:18.138414 21753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:18.138438 21753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:58:18.139799 21753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:42267
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=32991
--tserver_master_addrs=127.7.37.254:36099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:18.139982 21753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:18.140161 21753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:18.144590 21762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:18.144598 21763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:18.144732 21753 server_base.cc:1034] running on GCE node
W20260926 14:58:18.144779 21765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:18.144942 21753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:18.145154 21753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:18.146288 21753 hybrid_clock.cc:648] HybridClock initialized: now 1790434698146278 us; error 25 us; skew 500 ppm
Sep 26 14:58:18 dist-test-slave-f9xb krb5kdc[20983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434698, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:18.148437 21753 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:58:18.148792 21753 webserver.cc:460] Webserver started at http://127.7.37.194:32991/ using document root <none> and password file <none>
I20260926 14:58:18.148963 21753 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:18.149019 21753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:18.150147 21753 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 14:58:18.150622 21772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:18.150760 21753 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:18.150833 21753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50"
format_stamp: "Formatted at 2026-09-26 14:58:17 on dist-test-slave-f9xb"
server_key: "301342a95ebbcc84d110f65e3b4f9287"
server_key_iv: "f09d8a14d76d91b0d2a5e564545b16c1"
server_key_version: "encryptionkey@0"
I20260926 14:58:18.151098 21753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:18.164022 21753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:18.165548 21753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:18.165661 21753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:18.165831 21753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:18.166075 21753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:58:18.166106 21753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:18.166131 21753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:58:18.166154 21753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:18.171335 21753 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:42267
I20260926 14:58:18.171386 21885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:42267 every 8 connection(s)
I20260926 14:58:18.171650 21753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:58:18.172364 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 21753
I20260926 14:58:18.172612 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:38377
--local_ip_for_outbound_sockets=127.7.37.195
--tserver_master_addrs=127.7.37.254:36099
--webserver_port=45035
--webserver_interface=127.7.37.195
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 14:58:18 dist-test-slave-f9xb krb5kdc[20983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434698, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:18.179952 21750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.172208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:35775 (local address 127.7.37.254:36099)
0926 14:58:18.172254 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:18.172256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:18.172487 (+ 231us) server_negotiation.cc:407] Connection header received
0926 14:58:18.172950 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:18.172951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:18.172963 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:18.173030 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:18.173830 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.174318 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.175221 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.175433 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.175485 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:18.177523 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:18.177524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:18.177525 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:18.177532 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:18.178778 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:18.179109 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:18.179110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:18.179110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:18.179161 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:18.179357 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:18.179358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:18.179358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:18.179464 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:18.179517 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:18.179765 (+ 248us) server_negotiation.cc:299] Negotiation successful
0926 14:58:18.179862 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 14:58:18.180101 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.172187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:35775)
0926 14:58:18.172369 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:18.172398 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:18.172862 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:18.173075 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:18.173078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:18.173573 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:18.173739 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:18.173743 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.174400 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.174401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:18.175144 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:18.175146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.175497 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.175498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:18.175613 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:18.176057 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:18.176065 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:18.177442 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:18.178848 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:18.178854 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:18.178856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:18.179048 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:18.179212 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:18.179212 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:18.179213 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:18.179306 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:18.179514 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:18.179520 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:18.179640 (+ 120us) client_negotiation.cc:769] Sending connection context
0926 14:58:18.179720 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 14:58:18.179871 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":120,"threads_started":1}
I20260926 14:58:18.180506 21886 heartbeater.cc:347] Connected to a master server at 127.7.37.254:36099
I20260926 14:58:18.180604 21886 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:18.180794 21886 heartbeater.cc:511] Master 127.7.37.254:36099 requested a full tablet report, sending...
I20260926 14:58:18.181133 21560 ts_manager.cc:194] Registered new tserver with Master: fe0eb2dcfa8c4a11b055b11db8a96d50 (127.7.37.194:42267)
I20260926 14:58:18.181488 21560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:35775
W20260926 14:58:18.249732 21889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:18.249897 21889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:18.249917 21889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:18.251045 21889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:18.251083 21889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:18.251111 21889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:58:18.252497 21889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:38377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=45035
--tserver_master_addrs=127.7.37.254:36099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:18.252681 21889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:18.252864 21889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:18.257315 21895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:18.257328 21897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:18.257361 21894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:18.257364 21889 server_base.cc:1034] running on GCE node
I20260926 14:58:18.257702 21889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:18.257895 21889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:18.259020 21889 hybrid_clock.cc:648] HybridClock initialized: now 1790434698259006 us; error 26 us; skew 500 ppm
Sep 26 14:58:18 dist-test-slave-f9xb krb5kdc[20983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434698, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:18.261103 21889 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:58:18.261461 21889 webserver.cc:460] Webserver started at http://127.7.37.195:45035/ using document root <none> and password file <none>
I20260926 14:58:18.261631 21889 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:18.261686 21889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:18.262804 21889 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:58:18.263262 21904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:18.263417 21889 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:18.263490 21889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "cfbb02f44e964cc3870e342766ab2911"
format_stamp: "Formatted at 2026-09-26 14:58:17 on dist-test-slave-f9xb"
server_key: "638ba429fedbb853ac039097d2f5e98d"
server_key_iv: "76514744c2873f38823412d51f258432"
server_key_version: "encryptionkey@0"
I20260926 14:58:18.263780 21889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:18.282915 21889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:18.284508 21889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:18.284618 21889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:18.284791 21889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:18.285029 21889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:58:18.285058 21889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:18.285081 21889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:58:18.285106 21889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:18.290086 21889 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:38377
I20260926 14:58:18.290144 22017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:38377 every 8 connection(s)
I20260926 14:58:18.290393 21889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
Sep 26 14:58:18 dist-test-slave-f9xb krb5kdc[20983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434698, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:18.296794 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 21889
I20260926 14:58:18.298501 21750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.290937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:46713 (local address 127.7.37.254:36099)
0926 14:58:18.290985 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:18.290987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:18.291180 (+ 193us) server_negotiation.cc:407] Connection header received
0926 14:58:18.291586 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:18.291587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:18.291599 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:18.291647 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:18.292484 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.292968 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.293878 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.294075 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.294129 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:18.295916 (+ 1787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:18.295918 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:18.295918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:18.295925 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:18.297225 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:18.297618 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:18.297619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:18.297619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:18.297678 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:18.297920 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:18.297921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:18.297921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:18.298036 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:18.298090 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:18.298338 (+ 248us) server_negotiation.cc:299] Negotiation successful
0926 14:58:18.298413 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:58:18.298666 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.290936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:46713)
0926 14:58:18.291070 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:18.291099 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:18.291497 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:18.291704 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:18.291707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:18.292218 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:18.292397 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:18.292401 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.293041 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.293042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:18.293796 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:18.293798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.294137 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.294138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:18.294242 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:18.294639 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:18.294647 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:18.295841 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:18.297280 (+ 1439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:18.297286 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:18.297288 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:18.297556 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:18.297763 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:18.297763 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:18.297763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:18.297867 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:18.298088 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:18.298093 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:18.298211 (+ 118us) client_negotiation.cc:769] Sending connection context
0926 14:58:18.298305 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 14:58:18.298461 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":77,"threads_started":1}
I20260926 14:58:18.299086 22018 heartbeater.cc:347] Connected to a master server at 127.7.37.254:36099
I20260926 14:58:18.299170 22018 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:18.299319 22018 heartbeater.cc:511] Master 127.7.37.254:36099 requested a full tablet report, sending...
I20260926 14:58:18.299655 21560 ts_manager.cc:194] Registered new tserver with Master: cfbb02f44e964cc3870e342766ab2911 (127.7.37.195:38377)
I20260926 14:58:18.300072 21560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:46713
I20260926 14:58:18.302816 22022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.297121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53118 (local address 127.7.37.254:36099)
0926 14:58:18.297222 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:18.297224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:18.297235 (+ 11us) server_negotiation.cc:407] Connection header received
0926 14:58:18.297309 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:18.297310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:18.297321 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:18.297391 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:18.297924 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.298432 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.299385 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.299650 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.299706 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:18.300906 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:18.300907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:18.300907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:18.300914 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:18.301902 (+ 988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:18.302182 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:18.302183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:18.302183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:18.302236 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:18.302390 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:18.302391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:18.302391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:18.302487 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:18.302634 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:18.302667 (+ 33us) server_negotiation.cc:299] Negotiation successful
0926 14:58:18.302733 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":65,"threads_started":1}
I20260926 14:58:18.303360 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 14:58:18.305511 22022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.303628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53122 (local address 127.7.37.254:36099)
0926 14:58:18.303659 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:18.303661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:18.303720 (+ 59us) server_negotiation.cc:407] Connection header received
0926 14:58:18.303810 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:18.303810 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:18.303837 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:18.303887 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:58:18.303956 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.304382 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.305012 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.305177 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.305213 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:18.305306 (+ 93us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:58:18.305394 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:58:18.305443 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:18.305454 (+ 11us) server_negotiation.cc:299] Negotiation successful
0926 14:58:18.305469 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 14:58:18.305728 21560 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:53122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:58:18.307238 21560 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:58:18.309079 21667 tablet_service.cc:1467] Processing CreateTablet for tablet c42f8918dc934772b39f7d5c7fab2667 (DEFAULT_TABLE table=test-table [id=26054593833e4e56af43e3d1777dfd48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:58:18.309252 21667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c42f8918dc934772b39f7d5c7fab2667. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:18.310668 21640 tablet_bootstrap.cc:492] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d: Bootstrap starting.
I20260926 14:58:18.310964 21640 tablet_bootstrap.cc:654] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:18.311503 21640 tablet_bootstrap.cc:492] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d: No bootstrap required, opened a new log
I20260926 14:58:18.311545 21640 ts_tablet_manager.cc:1397] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:58:18.311858 21640 raft_consensus.cc:348] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } }
I20260926 14:58:18.311923 21640 raft_consensus.cc:374] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:18.311967 21640 raft_consensus.cc:729] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db551780aa8941bb825d07a910af1d7d, State: Initialized, Role: FOLLOWER
I20260926 14:58:18.312008 21640 consensus_queue.cc:260] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } }
I20260926 14:58:18.312098 21640 ts_tablet_manager.cc:1428] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:58:18.312320 21755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.308771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:42267 (local address 127.0.0.1:43414)
0926 14:58:18.308865 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:18.308884 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:18.308918 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:18.309293 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:18.309294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:18.309296 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:18.309329 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:18.309331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.310197 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.310198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:18.311196 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:18.311198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.312106 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.312107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:18.312174 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:18.312192 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 14:58:18.312221 (+ 29us) client_negotiation.cc:240] Negotiation successful
0926 14:58:18.312261 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260926 14:58:18.312619 22024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.308808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:38377 (local address 127.0.0.1:58498)
0926 14:58:18.308935 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:18.308944 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:18.308990 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:18.309398 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:18.309399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:18.309400 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:18.309428 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:18.309428 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.310536 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.310536 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:18.311537 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:18.311539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.312418 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.312419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:18.312479 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:18.312482 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:58:18.312525 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 14:58:18.312562 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260926 14:58:18.312682 22025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.308862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43414 (local address 127.7.37.194:42267)
0926 14:58:18.309160 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:18.309163 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:18.309171 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:58:18.309200 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:18.309201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:18.309213 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:18.309279 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:18.309394 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.310112 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.311284 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.312014 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.312489 (+ 475us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:18.312549 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:18.312572 (+ 23us) server_negotiation.cc:299] Negotiation successful
0926 14:58:18.312609 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":82,"threads_started":1}
I20260926 14:58:18.312912 22026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.309019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58498 (local address 127.7.37.195:38377)
0926 14:58:18.309147 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:18.309151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:18.309161 (+ 10us) server_negotiation.cc:407] Connection header received
0926 14:58:18.309189 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:18.309191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:18.309203 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:18.309279 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:18.309487 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.310438 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.311614 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.312282 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.312335 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:18.312385 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:18.312801 (+ 416us) server_negotiation.cc:299] Negotiation successful
0926 14:58:18.312848 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260926 14:58:18.313230 21820 tablet_service.cc:1467] Processing CreateTablet for tablet c42f8918dc934772b39f7d5c7fab2667 (DEFAULT_TABLE table=test-table [id=26054593833e4e56af43e3d1777dfd48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:58:18.313448 21952 tablet_service.cc:1467] Processing CreateTablet for tablet c42f8918dc934772b39f7d5c7fab2667 (DEFAULT_TABLE table=test-table [id=26054593833e4e56af43e3d1777dfd48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:58:18.313544 21820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c42f8918dc934772b39f7d5c7fab2667. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:18.313707 21952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c42f8918dc934772b39f7d5c7fab2667. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:18.316690 22028 tablet_bootstrap.cc:492] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50: Bootstrap starting.
I20260926 14:58:18.317339 22028 tablet_bootstrap.cc:654] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:18.317761 22028 log.cc:824] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:18.318013 22029 tablet_bootstrap.cc:492] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911: Bootstrap starting.
I20260926 14:58:18.318315 22028 tablet_bootstrap.cc:492] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50: No bootstrap required, opened a new log
I20260926 14:58:18.318369 22028 ts_tablet_manager.cc:1397] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:58:18.319687 22028 raft_consensus.cc:348] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } }
I20260926 14:58:18.319839 22028 raft_consensus.cc:374] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:18.319890 22028 raft_consensus.cc:729] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe0eb2dcfa8c4a11b055b11db8a96d50, State: Initialized, Role: FOLLOWER
I20260926 14:58:18.319993 22028 consensus_queue.cc:260] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } }
I20260926 14:58:18.320217 22028 ts_tablet_manager.cc:1428] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:58:18.320228 21886 heartbeater.cc:503] Master 127.7.37.254:36099 was elected leader, sending a full tablet report...
I20260926 14:58:18.320264 22029 tablet_bootstrap.cc:654] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:18.320719 22029 log.cc:824] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:18.321334 22029 tablet_bootstrap.cc:492] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911: No bootstrap required, opened a new log
I20260926 14:58:18.321385 22029 ts_tablet_manager.cc:1397] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 14:58:18.322615 22029 raft_consensus.cc:348] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } }
I20260926 14:58:18.322732 22029 raft_consensus.cc:374] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:18.322752 22029 raft_consensus.cc:729] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfbb02f44e964cc3870e342766ab2911, State: Initialized, Role: FOLLOWER
I20260926 14:58:18.322826 22029 consensus_queue.cc:260] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } }
I20260926 14:58:18.323024 22029 ts_tablet_manager.cc:1428] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:58:18.323052 22018 heartbeater.cc:503] Master 127.7.37.254:36099 was elected leader, sending a full tablet report...
I20260926 14:58:18.351223 22031 raft_consensus.cc:482] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:58:18.351289 22031 raft_consensus.cc:504] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } }
I20260926 14:58:18.351554 22031 leader_election.cc:288] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfbb02f44e964cc3870e342766ab2911 (127.7.37.195:38377), db551780aa8941bb825d07a910af1d7d (127.7.37.193:33929)
I20260926 14:58:18.354719 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.351672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:38377 (local address 127.7.37.194:38571)
0926 14:58:18.351704 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:18.351713 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:18.351774 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:18.351925 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:18.351926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:18.351928 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:18.351984 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:18.351985 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.352801 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.352802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:18.353910 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:18.353912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.354531 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.354531 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:18.354583 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:18.354586 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:58:18.354621 (+ 35us) client_negotiation.cc:240] Negotiation successful
0926 14:58:18.354655 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 14:58:18.354722 22026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.351719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:38571 (local address 127.7.37.195:38377)
0926 14:58:18.351777 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:18.351779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:18.351786 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:58:18.351850 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:18.351851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:18.351860 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:18.351909 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:18.352042 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.352725 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.353983 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.354470 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.354535 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:18.354541 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:18.354641 (+ 100us) server_negotiation.cc:299] Negotiation successful
0926 14:58:18.354670 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 14:58:18.355016 21972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c42f8918dc934772b39f7d5c7fab2667" candidate_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfbb02f44e964cc3870e342766ab2911" is_pre_election: true
I20260926 14:58:18.355094 22034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.351672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:33929 (local address 127.7.37.194:34085)
0926 14:58:18.351853 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:18.351862 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:18.351941 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:18.352088 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:18.352088 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:18.352090 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:18.352136 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:18.352136 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.352981 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.352982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:18.354064 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:18.354066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.354754 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.354857 (+ 103us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:18.354940 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:18.354944 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:58:18.354992 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 14:58:18.355025 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":37,"thread_start_us":83,"threads_started":1}
I20260926 14:58:18.355099 21756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.351721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:34085 (local address 127.7.37.193:33929)
0926 14:58:18.351773 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:18.351775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:18.351918 (+ 143us) server_negotiation.cc:407] Connection header received
0926 14:58:18.352005 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:18.352006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:18.352023 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:18.352081 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:18.352190 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.352925 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.354121 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.354693 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.354752 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:18.354812 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:18.355007 (+ 195us) server_negotiation.cc:299] Negotiation successful
0926 14:58:18.355045 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 14:58:18.355111 21972 raft_consensus.cc:2454] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe0eb2dcfa8c4a11b055b11db8a96d50 in term 0.
I20260926 14:58:18.355307 21776 leader_election.cc:302] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cfbb02f44e964cc3870e342766ab2911, fe0eb2dcfa8c4a11b055b11db8a96d50; no voters:
I20260926 14:58:18.355347 21690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c42f8918dc934772b39f7d5c7fab2667" candidate_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db551780aa8941bb825d07a910af1d7d" is_pre_election: true
I20260926 14:58:18.355432 22031 raft_consensus.cc:2781] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:58:18.355489 22031 raft_consensus.cc:482] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:58:18.355516 22031 raft_consensus.cc:3037] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:18.355499 21690 raft_consensus.cc:2454] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe0eb2dcfa8c4a11b055b11db8a96d50 in term 0.
I20260926 14:58:18.356137 22031 raft_consensus.cc:504] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } }
I20260926 14:58:18.356231 22031 leader_election.cc:288] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [CANDIDATE]: Term 1 election: Requested vote from peers cfbb02f44e964cc3870e342766ab2911 (127.7.37.195:38377), db551780aa8941bb825d07a910af1d7d (127.7.37.193:33929)
I20260926 14:58:18.356380 21972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c42f8918dc934772b39f7d5c7fab2667" candidate_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfbb02f44e964cc3870e342766ab2911"
I20260926 14:58:18.356393 21690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c42f8918dc934772b39f7d5c7fab2667" candidate_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db551780aa8941bb825d07a910af1d7d"
I20260926 14:58:18.356451 21972 raft_consensus.cc:3037] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:18.356453 21690 raft_consensus.cc:3037] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:18.357151 21972 raft_consensus.cc:2454] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe0eb2dcfa8c4a11b055b11db8a96d50 in term 1.
I20260926 14:58:18.357151 21690 raft_consensus.cc:2454] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe0eb2dcfa8c4a11b055b11db8a96d50 in term 1.
I20260926 14:58:18.357306 21776 leader_election.cc:302] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cfbb02f44e964cc3870e342766ab2911, fe0eb2dcfa8c4a11b055b11db8a96d50; no voters:
I20260926 14:58:18.357393 22031 raft_consensus.cc:2781] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:58:18.357512 22031 raft_consensus.cc:686] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 1 LEADER]: Becoming Leader. State: Replica: fe0eb2dcfa8c4a11b055b11db8a96d50, State: Running, Role: LEADER
I20260926 14:58:18.357579 22031 consensus_queue.cc:237] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } }
I20260926 14:58:18.358198 21560 catalog_manager.cc:5495] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to fe0eb2dcfa8c4a11b055b11db8a96d50 (127.7.37.194). New cstate: current_term: 1 leader_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } health_report { overall_health: UNKNOWN } } }
I20260926 14:58:18.373322 22025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.371561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43420 (local address 127.7.37.194:42267)
0926 14:58:18.371593 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:18.371595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:18.371601 (+ 6us) server_negotiation.cc:407] Connection header received
0926 14:58:18.371650 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:18.371651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:18.371662 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:18.371713 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:58:18.371814 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.372215 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.372825 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:18.372988 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:18.373027 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:18.373121 (+ 94us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:58:18.373193 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:58:18.373239 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:18.373254 (+ 15us) server_negotiation.cc:299] Negotiation successful
0926 14:58:18.373266 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260926 14:58:18.374610 21972 raft_consensus.cc:1261] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911 [term 1 FOLLOWER]: Refusing update from remote peer fe0eb2dcfa8c4a11b055b11db8a96d50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:58:18.374611 21690 raft_consensus.cc:1261] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Refusing update from remote peer fe0eb2dcfa8c4a11b055b11db8a96d50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:58:18.374888 22035 consensus_queue.cc:1035] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:58:18.375000 22031 consensus_queue.cc:1035] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:58:18.378638 21560 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:53122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:58:18.378715 21560 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:53122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:58:18.379230 21560 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3: Sending DeleteTablet for 3 replicas of tablet c42f8918dc934772b39f7d5c7fab2667
I20260926 14:58:18.379496 21667 tablet_service.cc:1514] Processing DeleteTablet for tablet c42f8918dc934772b39f7d5c7fab2667 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:58:18 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:51532
I20260926 14:58:18.379508 21820 tablet_service.cc:1514] Processing DeleteTablet for tablet c42f8918dc934772b39f7d5c7fab2667 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:58:18 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:43414
I20260926 14:58:18.379585 21757 tablet_replica.cc:330] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d: stopping tablet replica
I20260926 14:58:18.379626 21757 raft_consensus.cc:2229] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:58:18.379673 21757 raft_consensus.cc:2258] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:58:18.379649 21952 tablet_service.cc:1514] Processing DeleteTablet for tablet c42f8918dc934772b39f7d5c7fab2667 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:58:18 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:58498
I20260926 14:58:18.379830 22048 tablet_replica.cc:330] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911: stopping tablet replica
I20260926 14:58:18.379818 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 21598
I20260926 14:58:18.379946 21757 ts_tablet_manager.cc:1905] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:58:18.379731 22047 tablet_replica.cc:330] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50: stopping tablet replica
I20260926 14:58:18.380764 22047 raft_consensus.cc:2229] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:58:18.380856 22047 raft_consensus.cc:2258] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:58:18.381162 22047 ts_tablet_manager.cc:1905] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:58:18.382424 22047 ts_tablet_manager.cc:1918] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:58:18.382481 22047 log.cc:1196] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/c42f8918dc934772b39f7d5c7fab2667
I20260926 14:58:18.382700 22047 ts_tablet_manager.cc:1939] T c42f8918dc934772b39f7d5c7fab2667 P fe0eb2dcfa8c4a11b055b11db8a96d50: Deleting consensus metadata
I20260926 14:58:18.383009 21545 catalog_manager.cc:4841] TS fe0eb2dcfa8c4a11b055b11db8a96d50 (127.7.37.194:42267): tablet c42f8918dc934772b39f7d5c7fab2667 (table test-table [id=26054593833e4e56af43e3d1777dfd48]) successfully deleted
I20260926 14:58:18.384135 22048 raft_consensus.cc:2229] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:58:18.384240 22048 raft_consensus.cc:2258] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:58:18.384552 22048 ts_tablet_manager.cc:1905] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:58:18.385600 22048 ts_tablet_manager.cc:1918] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:58:18.385660 22048 log.cc:1196] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/c42f8918dc934772b39f7d5c7fab2667
W20260926 14:58:18.385699 21544 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.37.193:33929 (error 108)
I20260926 14:58:18.385903 22048 ts_tablet_manager.cc:1939] T c42f8918dc934772b39f7d5c7fab2667 P cfbb02f44e964cc3870e342766ab2911: Deleting consensus metadata
I20260926 14:58:18.386140 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 21753
I20260926 14:58:18.386777 21547 catalog_manager.cc:4841] TS cfbb02f44e964cc3870e342766ab2911 (127.7.37.195:38377): tablet c42f8918dc934772b39f7d5c7fab2667 (table test-table [id=26054593833e4e56af43e3d1777dfd48]) successfully deleted
I20260926 14:58:18.387559 22024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.386693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:33929 (local address 127.0.0.1:51538)
0926 14:58:18.387503 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:18.387537 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.193:33929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":793}
W20260926 14:58:18.387666 21544 catalog_manager.cc:4584] TS db551780aa8941bb825d07a910af1d7d (127.7.37.193:33929): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c42f8918dc934772b39f7d5c7fab2667: Network error: Client connection negotiation failed: client connection to 127.7.37.193:33929: connect: Connection refused (error 111)
I20260926 14:58:18.390630 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 21889
I20260926 14:58:18.394330 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 21527
I20260926 14:58:18.398260 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36099
--webserver_interface=127.7.37.254
--webserver_port=34709
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:36099
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:58:18.472697 22050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:18.472877 22050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:18.472896 22050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:18.473971 22050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:58:18.474006 22050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:18.474018 22050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:18.474040 22050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:58:18.474061 22050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:58:18.475440 22050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:36099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=34709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:18.475641 22050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:18.475845 22050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:18.479962 22056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:18.479971 22055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:18.479995 22050 server_base.cc:1034] running on GCE node
W20260926 14:58:18.480013 22058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:18.480310 22050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:18.480557 22050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:18.481680 22050 hybrid_clock.cc:648] HybridClock initialized: now 1790434698481669 us; error 25 us; skew 500 ppm
Sep 26 14:58:18 dist-test-slave-f9xb krb5kdc[20983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434698, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:18.488725 22050 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:58:18.489104 22050 webserver.cc:460] Webserver started at http://127.7.37.254:34709/ using document root <none> and password file <none>
I20260926 14:58:18.489289 22050 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:18.489333 22050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:18.490477 22050 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:58:18.491010 22065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:18.491168 22050 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:58:18.491240 22050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "d918869b52414516bf5ccaa937ec33b3"
format_stamp: "Formatted at 2026-09-26 14:58:16 on dist-test-slave-f9xb"
server_key: "af7ea4765c5e19eff5566bd087ac2356"
server_key_iv: "ac5083022859dda767ebf91def893dd8"
server_key_version: "encryptionkey@0"
I20260926 14:58:18.491509 22050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:18.509737 22050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:18.511262 22050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:18.511381 22050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:18.514864 22050 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:36099
I20260926 14:58:18.514892 22117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:36099 every 8 connection(s)
I20260926 14:58:18.515179 22050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:58:18.516489 22118 sys_catalog.cc:263] Verifying existing consensus state
I20260926 14:58:18.517015 22118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3: Bootstrap starting.
I20260926 14:58:18.522102 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 22050
I20260926 14:58:18.522334 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:33929
--local_ip_for_outbound_sockets=127.7.37.193
--tserver_master_addrs=127.7.37.254:36099
--webserver_port=34023
--webserver_interface=127.7.37.193
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 14:58:18.524576 22118 log.cc:824] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:18.526513 22118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 14:58:18.526742 22118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3: Bootstrap complete.
I20260926 14:58:18.528669 22118 raft_consensus.cc:348] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d918869b52414516bf5ccaa937ec33b3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36099 } }
I20260926 14:58:18.528877 22118 raft_consensus.cc:729] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d918869b52414516bf5ccaa937ec33b3, State: Initialized, Role: FOLLOWER
I20260926 14:58:18.528997 22118 consensus_queue.cc:260] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d918869b52414516bf5ccaa937ec33b3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36099 } }
I20260926 14:58:18.529052 22118 raft_consensus.cc:388] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:58:18.529086 22118 raft_consensus.cc:482] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:58:18.529129 22118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 2 FOLLOWER]: Advancing to term 3
I20260926 14:58:18.529994 22118 raft_consensus.cc:504] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d918869b52414516bf5ccaa937ec33b3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36099 } }
I20260926 14:58:18.530073 22118 leader_election.cc:302] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d918869b52414516bf5ccaa937ec33b3; no voters:
I20260926 14:58:18.530201 22118 leader_election.cc:288] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 14:58:18.530244 22122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 14:58:18.530357 22122 raft_consensus.cc:686] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [term 3 LEADER]: Becoming Leader. State: Replica: d918869b52414516bf5ccaa937ec33b3, State: Running, Role: LEADER
I20260926 14:58:18.530427 22118 sys_catalog.cc:564] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:58:18.530444 22122 consensus_queue.cc:237] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d918869b52414516bf5ccaa937ec33b3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36099 } }
I20260926 14:58:18.530696 22122 sys_catalog.cc:455] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d918869b52414516bf5ccaa937ec33b3. Latest consensus state: current_term: 3 leader_uuid: "d918869b52414516bf5ccaa937ec33b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d918869b52414516bf5ccaa937ec33b3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36099 } } }
I20260926 14:58:18.530771 22122 sys_catalog.cc:458] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [sys.catalog]: This master's current role is: LEADER
I20260926 14:58:18.531100 22127 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:58:18.531116 22123 sys_catalog.cc:455] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d918869b52414516bf5ccaa937ec33b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d918869b52414516bf5ccaa937ec33b3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36099 } } }
I20260926 14:58:18.531277 22123 sys_catalog.cc:458] T 00000000000000000000000000000000 P d918869b52414516bf5ccaa937ec33b3 [sys.catalog]: This master's current role is: LEADER
I20260926 14:58:18.531890 22127 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:58:18.531991 22127 catalog_manager.cc:1259] Loaded cluster ID: e7aa72c553ac45c28c8292b407daffe5
I20260926 14:58:18.532030 22127 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260926 14:58:18.532374 22127 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f47c2d999ad google::LogMessage::Fail() at ??:0
@ 0x7f47c2d9c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f47c2d994de google::LogMessage::Flush() at ??:0
@ 0x7f47c2d9aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f47cacba0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f47cacc38b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f47c4c670d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f47c4c5c954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f47c5ac06db start_thread at ??:0
@ 0x7f47c189761f clone at ??:0
W20260926 14:58:18.606627 22120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:18.606806 22120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:18.606837 22120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:18.608037 22120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:18.608078 22120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:18.608114 22120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:58:18.609490 22120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:33929
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=34023
--tserver_master_addrs=127.7.37.254:36099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:18.609704 22120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:18.609889 22120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:18.614269 22154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:18.614326 22155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:18.614419 22120 server_base.cc:1034] running on GCE node
W20260926 14:58:18.614326 22157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:18.614631 22120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:18.614830 22120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:18.615976 22120 hybrid_clock.cc:648] HybridClock initialized: now 1790434698615953 us; error 37 us; skew 500 ppm
Sep 26 14:58:18 dist-test-slave-f9xb krb5kdc[20983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434698, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:18.618104 22120 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:58:18.618489 22120 webserver.cc:460] Webserver started at http://127.7.37.193:34023/ using document root <none> and password file <none>
I20260926 14:58:18.618674 22120 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:18.618718 22120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:18.619979 22120 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:58:18.620669 22164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:18.620823 22120 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:58:18.620896 22120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "db551780aa8941bb825d07a910af1d7d"
format_stamp: "Formatted at 2026-09-26 14:58:17 on dist-test-slave-f9xb"
server_key: "e0a7f4d49066216a53849fe80818bfa6"
server_key_iv: "ac247eefae177f7acad12b617cde873c"
server_key_version: "encryptionkey@0"
I20260926 14:58:18.621160 22120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:18.647529 22120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:18.649435 22120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:18.649556 22120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:18.649729 22120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:18.650107 22171 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 14:58:18.651825 22120 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 14:58:18.651866 22120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 14:58:18.651896 22120 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 14:58:18.652438 22120 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 14:58:18.652468 22120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 14:58:18.652555 22171 tablet_bootstrap.cc:492] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d: Bootstrap starting.
I20260926 14:58:18.654232 22171 log.cc:824] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:18.657032 22171 tablet_bootstrap.cc:492] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 14:58:18.657320 22171 tablet_bootstrap.cc:492] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d: Bootstrap complete.
I20260926 14:58:18.657465 22171 ts_tablet_manager.cc:1397] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 14:58:18.658783 22120 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:33929
I20260926 14:58:18.658876 22279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:33929 every 8 connection(s)
I20260926 14:58:18.659101 22120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:58:18.659181 22171 raft_consensus.cc:348] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } }
I20260926 14:58:18.659888 22171 raft_consensus.cc:729] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db551780aa8941bb825d07a910af1d7d, State: Initialized, Role: FOLLOWER
I20260926 14:58:18.660018 22171 consensus_queue.cc:260] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } }
I20260926 14:58:18.660260 22171 ts_tablet_manager.cc:1428] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 14:58:18.666610 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 22120
I20260926 14:58:18.666883 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:42267
--local_ip_for_outbound_sockets=127.7.37.194
--tserver_master_addrs=127.7.37.254:36099
--webserver_port=32991
--webserver_interface=127.7.37.194
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:58:18.730779 22282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:58:18.659633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:53657)
0926 14:58:18.659790 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:18.659832 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:18.660475 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:18.730606 (+ 70131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
W20260926 14:58:18.730997 22280 heartbeater.cc:644] Failed to heartbeat to 127.7.37.254:36099 (0 consecutive failures): Network error: Failed to ping master at 127.7.37.254:36099: Client connection negotiation failed: client connection to 127.7.37.254:36099: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 14:58:18.731287 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.731187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:44347)
0926 14:58:18.731230 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:18.731266 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260926 14:58:18.731596 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.731480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:44205)
0926 14:58:18.731528 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:18.731567 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 14:58:18.731720 22280 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 14:58:18.755869 22284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:18.756031 22284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:18.756052 22284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:18.757160 22284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:18.757195 22284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:18.757223 22284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:58:18.758584 22284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:42267
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=32991
--tserver_master_addrs=127.7.37.254:36099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:18.758776 22284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:18.758966 22284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:18.763447 22289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:18.763447 22284 server_base.cc:1034] running on GCE node
W20260926 14:58:18.763428 22290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:18.763563 22292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:18.763836 22284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:18.764063 22284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:18.765198 22284 hybrid_clock.cc:648] HybridClock initialized: now 1790434698765185 us; error 27 us; skew 500 ppm
Sep 26 14:58:18 dist-test-slave-f9xb krb5kdc[20983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434698, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:18.767295 22284 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:58:18.767650 22284 webserver.cc:460] Webserver started at http://127.7.37.194:32991/ using document root <none> and password file <none>
I20260926 14:58:18.767892 22284 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:18.767940 22284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:18.769133 22284 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:58:18.769642 22299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:18.769794 22284 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:18.769878 22284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50"
format_stamp: "Formatted at 2026-09-26 14:58:17 on dist-test-slave-f9xb"
server_key: "301342a95ebbcc84d110f65e3b4f9287"
server_key_iv: "f09d8a14d76d91b0d2a5e564545b16c1"
server_key_version: "encryptionkey@0"
I20260926 14:58:18.770151 22284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:18.779681 22284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:18.781265 22284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:18.781383 22284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:18.781554 22284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:18.781796 22284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:58:18.781826 22284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:18.781859 22284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:58:18.781880 22284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:18.786978 22284 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:42267
I20260926 14:58:18.787034 22412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:42267 every 8 connection(s)
I20260926 14:58:18.787294 22284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:58:18.788197 22415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.787855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:49387)
0926 14:58:18.788011 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:18.788077 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":64,"threads_started":1}
W20260926 14:58:18.788546 22413 heartbeater.cc:644] Failed to heartbeat to 127.7.37.254:36099 (0 consecutive failures): Network error: Failed to ping master at 127.7.37.254:36099: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
I20260926 14:58:18.788759 22415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.788687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:39311)
0926 14:58:18.788721 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:18.788744 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260926 14:58:18.788947 22415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.788902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:45321)
0926 14:58:18.788921 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:18.788936 (+ 15us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260926 14:58:18.789024 22413 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 14:58:18.791678 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 22284
I20260926 14:58:18.791922 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:38377
--local_ip_for_outbound_sockets=127.7.37.195
--tserver_master_addrs=127.7.37.254:36099
--webserver_port=45035
--webserver_interface=127.7.37.195
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:58:18.866428 22416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:18.866593 22416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:18.866613 22416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:18.867697 22416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:18.867734 22416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:18.867784 22416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:58:18.869139 22416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:38377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=45035
--tserver_master_addrs=127.7.37.254:36099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:18.869340 22416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:18.869517 22416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:18.873939 22422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:18.873888 22424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:18.873901 22421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:18.874053 22416 server_base.cc:1034] running on GCE node
I20260926 14:58:18.874289 22416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:18.874513 22416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:18.875648 22416 hybrid_clock.cc:648] HybridClock initialized: now 1790434698875629 us; error 30 us; skew 500 ppm
Sep 26 14:58:18 dist-test-slave-f9xb krb5kdc[20983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434698, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:18.877810 22416 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:58:18.878157 22416 webserver.cc:460] Webserver started at http://127.7.37.195:45035/ using document root <none> and password file <none>
I20260926 14:58:18.878324 22416 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:18.878368 22416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:18.879480 22416 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 14:58:18.879985 22431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:18.880107 22416 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:18.880178 22416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "cfbb02f44e964cc3870e342766ab2911"
format_stamp: "Formatted at 2026-09-26 14:58:17 on dist-test-slave-f9xb"
server_key: "638ba429fedbb853ac039097d2f5e98d"
server_key_iv: "76514744c2873f38823412d51f258432"
server_key_version: "encryptionkey@0"
I20260926 14:58:18.880434 22416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:18.898814 22416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:18.900691 22416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:18.900806 22416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:18.900983 22416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:18.901218 22416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:58:18.901248 22416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:18.901273 22416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:58:18.901290 22416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:18.906316 22416 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:38377
I20260926 14:58:18.906380 22544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:38377 every 8 connection(s)
I20260926 14:58:18.906632 22416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:58:18.907449 22547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.907151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:39107)
0926 14:58:18.907276 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:18.907333 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":59,"threads_started":1}
W20260926 14:58:18.907858 22545 heartbeater.cc:644] Failed to heartbeat to 127.7.37.254:36099 (0 consecutive failures): Network error: Failed to ping master at 127.7.37.254:36099: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
I20260926 14:58:18.908090 22547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.908007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:48833)
0926 14:58:18.908046 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:18.908073 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260926 14:58:18.908280 22547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:18.908232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:57795)
0926 14:58:18.908251 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:18.908269 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260926 14:58:18.908354 22545 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 14:58:18.915771 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 22416
I20260926 14:58:19.732213 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:19.732029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:51589)
0926 14:58:19.732133 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:19.732160 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":50,"threads_started":1}
I20260926 14:58:19.789527 22550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:19.789315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:44065)
0926 14:58:19.789450 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:19.789482 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":58,"threads_started":1}
I20260926 14:58:19.908828 22551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:19.908630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:42249)
0926 14:58:19.908751 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:19.908782 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":53,"threads_started":1}
I20260926 14:58:20.472033 22552 raft_consensus.cc:482] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:58:20.472144 22552 raft_consensus.cc:504] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } }
I20260926 14:58:20.472463 22552 leader_election.cc:288] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fe0eb2dcfa8c4a11b055b11db8a96d50 (127.7.37.194:42267), cfbb02f44e964cc3870e342766ab2911 (127.7.37.195:38377)
Sep 26 14:58:20 dist-test-slave-f9xb krb5kdc[20983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434698, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.194@KRBTEST.COM
Sep 26 14:58:20 dist-test-slave-f9xb krb5kdc[20983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434698, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.195@KRBTEST.COM
I20260926 14:58:20.482340 22553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:20.472584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:38377 (local address 127.7.37.193:60069)
0926 14:58:20.472676 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:20.472685 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:20.472738 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:20.473529 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:20.473532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:20.473881 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:20.474211 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:20.474214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:20.475191 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:20.475192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:20.476029 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:20.476032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:20.476554 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:20.476555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:20.476667 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:20.477153 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:20.477160 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:20.478815 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:20.480693 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:20.480693 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:20.480694 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:20.481066 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:20.481309 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:20.481310 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:20.481310 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:20.481447 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:20.481849 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:20.481854 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:20.481992 (+ 138us) client_negotiation.cc:769] Sending connection context
0926 14:58:20.482088 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 14:58:20.482238 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"mutex_wait_us":4,"thread_start_us":43,"threads_started":1}
I20260926 14:58:20.482378 22555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:20.472669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:60069 (local address 127.7.37.195:38377)
0926 14:58:20.472843 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:20.472853 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:20.472871 (+ 18us) server_negotiation.cc:407] Connection header received
0926 14:58:20.473313 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:20.473316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:20.473367 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:20.473522 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:20.474306 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:20.475117 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:20.476107 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:20.476483 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:20.476572 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:20.478902 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:20.478905 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:20.478906 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:20.478916 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:20.480633 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:20.481142 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:20.481145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:20.481145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:20.481246 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:20.481507 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:20.481507 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:20.481507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:20.481784 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:20.481865 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:20.482151 (+ 286us) server_negotiation.cc:299] Negotiation successful
0926 14:58:20.482274 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":45,"threads_started":1}
I20260926 14:58:20.482393 22556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:20.472802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:56427 (local address 127.7.37.194:42267)
0926 14:58:20.472940 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:20.472950 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:20.472970 (+ 20us) server_negotiation.cc:407] Connection header received
0926 14:58:20.473362 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:20.473364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:20.473399 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:20.473516 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:20.474189 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:20.475025 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:20.476100 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:20.476485 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:20.476582 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:20.478572 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:20.478577 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:20.478577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:20.478588 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:20.480609 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:20.481142 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:20.481145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:20.481145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:20.481246 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:20.481507 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:20.481507 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:20.481507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:20.481785 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:20.481865 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:20.482152 (+ 287us) server_negotiation.cc:299] Negotiation successful
0926 14:58:20.482274 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":56,"threads_started":1}
I20260926 14:58:20.482911 22554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:20.472593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:42267 (local address 127.7.37.193:56427)
0926 14:58:20.472687 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:20.472696 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:20.472739 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:20.473530 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:20.473533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:20.473883 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:20.474087 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:20.474092 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:20.475112 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:20.475112 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:20.476030 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:20.476033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:20.476558 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:20.476559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:20.476669 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:20.477216 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:20.477221 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:20.478469 (+ 1248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:20.480683 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:20.480690 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:20.480692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:20.481066 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:20.481330 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:20.481331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:20.481331 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:20.481447 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:20.481849 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:20.481854 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:20.481992 (+ 138us) client_negotiation.cc:769] Sending connection context
0926 14:58:20.482088 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 14:58:20.482302 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":149,"thread_start_us":47,"threads_started":1}
W20260926 14:58:20.483352 22168 leader_election.cc:341] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cfbb02f44e964cc3870e342766ab2911 (127.7.37.195:38377): Not found: Tablet not found: c42f8918dc934772b39f7d5c7fab2667
W20260926 14:58:20.483511 22166 leader_election.cc:341] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fe0eb2dcfa8c4a11b055b11db8a96d50 (127.7.37.194:42267): Not found: Tablet not found: c42f8918dc934772b39f7d5c7fab2667
I20260926 14:58:20.483566 22166 leader_election.cc:302] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db551780aa8941bb825d07a910af1d7d; no voters: cfbb02f44e964cc3870e342766ab2911, fe0eb2dcfa8c4a11b055b11db8a96d50
I20260926 14:58:20.483659 22552 raft_consensus.cc:2726] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 14:58:20.732685 22554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:20.732578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:33333)
0926 14:58:20.732626 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:20.732652 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 14:58:20.790084 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:20.789883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:42167)
0926 14:58:20.789996 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:20.790037 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":52,"threads_started":1}
I20260926 14:58:20.909389 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:20.909189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:34553)
0926 14:58:20.909287 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:20.909330 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":52,"threads_started":1}
I20260926 14:58:21.733220 22559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:21.733042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:37911)
0926 14:58:21.733147 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:21.733178 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":58,"threads_started":1}
I20260926 14:58:21.790617 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:21.790450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:55793)
0926 14:58:21.790549 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:21.790581 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":54,"threads_started":1}
I20260926 14:58:21.909998 22561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:21.909785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:47283)
0926 14:58:21.909907 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:21.909943 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260926 14:58:22.275818 22562 raft_consensus.cc:482] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:58:22.275907 22562 raft_consensus.cc:504] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } }
I20260926 14:58:22.276041 22562 leader_election.cc:288] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fe0eb2dcfa8c4a11b055b11db8a96d50 (127.7.37.194:42267), cfbb02f44e964cc3870e342766ab2911 (127.7.37.195:38377)
W20260926 14:58:22.276400 22166 leader_election.cc:341] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fe0eb2dcfa8c4a11b055b11db8a96d50 (127.7.37.194:42267): Not found: Tablet not found: c42f8918dc934772b39f7d5c7fab2667
W20260926 14:58:22.276497 22168 leader_election.cc:341] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cfbb02f44e964cc3870e342766ab2911 (127.7.37.195:38377): Not found: Tablet not found: c42f8918dc934772b39f7d5c7fab2667
I20260926 14:58:22.276559 22168 leader_election.cc:302] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db551780aa8941bb825d07a910af1d7d; no voters: cfbb02f44e964cc3870e342766ab2911, fe0eb2dcfa8c4a11b055b11db8a96d50
I20260926 14:58:22.276633 22562 raft_consensus.cc:2726] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 14:58:22.733807 22563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:22.733634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:54283)
0926 14:58:22.733733 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:22.733765 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":54,"threads_started":1}
I20260926 14:58:22.791088 22564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:22.790910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:40755)
0926 14:58:22.791009 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:22.791039 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":49,"threads_started":1}
I20260926 14:58:22.910615 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:22.910405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:53237)
0926 14:58:22.910533 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:22.910565 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":72,"threads_started":1}
I20260926 14:58:23.734489 22566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:23.734251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:36649)
0926 14:58:23.734407 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:23.734443 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":81,"threads_started":1}
I20260926 14:58:23.791595 22567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:23.791419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:55817)
0926 14:58:23.791527 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:23.791558 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":48,"threads_started":1}
I20260926 14:58:23.911216 22568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:23.911016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:49357)
0926 14:58:23.911148 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:23.911181 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":57,"threads_started":1}
I20260926 14:58:24.006438 22569 raft_consensus.cc:482] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:58:24.006520 22569 raft_consensus.cc:504] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } }
I20260926 14:58:24.006681 22569 leader_election.cc:288] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fe0eb2dcfa8c4a11b055b11db8a96d50 (127.7.37.194:42267), cfbb02f44e964cc3870e342766ab2911 (127.7.37.195:38377)
W20260926 14:58:24.006973 22166 leader_election.cc:341] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fe0eb2dcfa8c4a11b055b11db8a96d50 (127.7.37.194:42267): Not found: Tablet not found: c42f8918dc934772b39f7d5c7fab2667
W20260926 14:58:24.007027 22168 leader_election.cc:341] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cfbb02f44e964cc3870e342766ab2911 (127.7.37.195:38377): Not found: Tablet not found: c42f8918dc934772b39f7d5c7fab2667
I20260926 14:58:24.007095 22168 leader_election.cc:302] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db551780aa8941bb825d07a910af1d7d; no voters: cfbb02f44e964cc3870e342766ab2911, fe0eb2dcfa8c4a11b055b11db8a96d50
I20260926 14:58:24.007184 22569 raft_consensus.cc:2726] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 14:58:24.735093 22570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:24.734902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:48329)
0926 14:58:24.735013 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:24.735046 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":56,"threads_started":1}
I20260926 14:58:24.792114 22571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:24.791909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:37267)
0926 14:58:24.792038 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:24.792074 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":55,"threads_started":1}
I20260926 14:58:24.911868 22572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:24.911635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:35575)
0926 14:58:24.911799 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:24.911832 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":68,"threads_started":1}
I20260926 14:58:25.735783 22573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:25.735522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:39677)
0926 14:58:25.735658 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:25.735701 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":58,"threads_started":1}
I20260926 14:58:25.792646 22574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:25.792461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:48523)
0926 14:58:25.792576 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:25.792608 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":52,"threads_started":1}
I20260926 14:58:25.912412 22575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:25.912214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:56777)
0926 14:58:25.912333 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:25.912368 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260926 14:58:26.736387 22576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:26.736167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:38083)
0926 14:58:26.736300 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:26.736337 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":57,"threads_started":1}
I20260926 14:58:26.793164 22577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:26.792997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:39411)
0926 14:58:26.793097 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:26.793130 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":56,"threads_started":1}
I20260926 14:58:26.913002 22578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:26.912816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:50461)
0926 14:58:26.912936 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:26.912966 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
I20260926 14:58:27.736928 22579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:27.736745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:34487)
0926 14:58:27.736859 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:27.736886 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":68,"threads_started":1}
I20260926 14:58:27.793744 22580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:27.793568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:44061)
0926 14:58:27.793678 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:27.793710 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":65,"threads_started":1}
I20260926 14:58:27.913585 22581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:27.913403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:52219)
0926 14:58:27.913519 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:27.913550 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":72,"threads_started":1}
I20260926 14:58:28.696236 22582 raft_consensus.cc:482] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:58:28.696336 22582 raft_consensus.cc:504] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } }
I20260926 14:58:28.696511 22582 leader_election.cc:288] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fe0eb2dcfa8c4a11b055b11db8a96d50 (127.7.37.194:42267), cfbb02f44e964cc3870e342766ab2911 (127.7.37.195:38377)
W20260926 14:58:28.696848 22168 leader_election.cc:341] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cfbb02f44e964cc3870e342766ab2911 (127.7.37.195:38377): Not found: Tablet not found: c42f8918dc934772b39f7d5c7fab2667
W20260926 14:58:28.696933 22166 leader_election.cc:341] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fe0eb2dcfa8c4a11b055b11db8a96d50 (127.7.37.194:42267): Not found: Tablet not found: c42f8918dc934772b39f7d5c7fab2667
I20260926 14:58:28.696977 22166 leader_election.cc:302] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db551780aa8941bb825d07a910af1d7d; no voters: cfbb02f44e964cc3870e342766ab2911, fe0eb2dcfa8c4a11b055b11db8a96d50
I20260926 14:58:28.697038 22582 raft_consensus.cc:2726] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 14:58:28.737473 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:28.737294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:59181)
0926 14:58:28.737398 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:28.737431 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":57,"threads_started":1}
I20260926 14:58:28.794278 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:28.794054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:41541)
0926 14:58:28.794190 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:28.794225 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260926 14:58:28.914157 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:28.913977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:54227)
0926 14:58:28.914090 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:28.914121 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":66,"threads_started":1}
I20260926 14:58:29.738018 22586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:29.737817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:39073)
0926 14:58:29.737941 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:29.737974 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":52,"threads_started":1}
I20260926 14:58:29.794837 22587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:29.794669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:33651)
0926 14:58:29.794772 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:29.794802 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
I20260926 14:58:29.914657 22588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:29.914497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:58761)
0926 14:58:29.914592 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:29.914622 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":55,"threads_started":1}
I20260926 14:58:30.738615 22589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:30.738405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:43021)
0926 14:58:30.738530 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:30.738566 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":54,"threads_started":1}
I20260926 14:58:30.795383 22590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:30.795179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:54589)
0926 14:58:30.795313 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:30.795346 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":53,"threads_started":1}
I20260926 14:58:30.915153 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:30.914983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:40219)
0926 14:58:30.915088 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:30.915116 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":64,"threads_started":1}
I20260926 14:58:31.739233 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:31.739041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:43631)
0926 14:58:31.739158 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:31.739190 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
I20260926 14:58:31.795948 22593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:31.795773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:57669)
0926 14:58:31.795879 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:31.795910 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":54,"threads_started":1}
I20260926 14:58:31.915731 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:31.915524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:41527)
0926 14:58:31.915657 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:31.915690 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
I20260926 14:58:32.080807 22595 raft_consensus.cc:482] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:58:32.080896 22595 raft_consensus.cc:504] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0eb2dcfa8c4a11b055b11db8a96d50" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42267 } } peers { permanent_uuid: "cfbb02f44e964cc3870e342766ab2911" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38377 } } peers { permanent_uuid: "db551780aa8941bb825d07a910af1d7d" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33929 } }
I20260926 14:58:32.081022 22595 leader_election.cc:288] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fe0eb2dcfa8c4a11b055b11db8a96d50 (127.7.37.194:42267), cfbb02f44e964cc3870e342766ab2911 (127.7.37.195:38377)
W20260926 14:58:32.081357 22166 leader_election.cc:341] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fe0eb2dcfa8c4a11b055b11db8a96d50 (127.7.37.194:42267): Not found: Tablet not found: c42f8918dc934772b39f7d5c7fab2667
W20260926 14:58:32.081450 22168 leader_election.cc:341] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cfbb02f44e964cc3870e342766ab2911 (127.7.37.195:38377): Not found: Tablet not found: c42f8918dc934772b39f7d5c7fab2667
I20260926 14:58:32.081511 22168 leader_election.cc:302] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db551780aa8941bb825d07a910af1d7d; no voters: cfbb02f44e964cc3870e342766ab2911, fe0eb2dcfa8c4a11b055b11db8a96d50
I20260926 14:58:32.081586 22595 raft_consensus.cc:2726] T c42f8918dc934772b39f7d5c7fab2667 P db551780aa8941bb825d07a910af1d7d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 14:58:32.739821 22596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:32.739611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:57083)
0926 14:58:32.739717 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:32.739768 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":56,"threads_started":1}
I20260926 14:58:32.796463 22597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:32.796285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:40187)
0926 14:58:32.796392 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:32.796422 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
I20260926 14:58:32.916326 22598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:32.916120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:42305)
0926 14:58:32.916257 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:32.916289 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":64,"threads_started":1}
I20260926 14:58:33.740412 22599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:33.740237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.193:35385)
0926 14:58:33.740345 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:33.740371 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260926 14:58:33.797021 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:33.796854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.194:51855)
0926 14:58:33.796954 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:33.796983 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":55,"threads_started":1}
I20260926 14:58:33.916941 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:33.916777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36099 (local address 127.7.37.195:36391)
0926 14:58:33.916876 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:33.916905 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":55,"threads_started":1}
I20260926 14:58:33.922853 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 22120
I20260926 14:58:33.926723 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 22284
I20260926 14:58:33.930184 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 22416
2026-09-26T14:58:33Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18197 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:58:33 dist-test-slave-f9xb krb5kdc[22604](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:58:33 dist-test-slave-f9xb krb5kdc[22604](info): set up 2 sockets
Sep 26 14:58:33 dist-test-slave-f9xb krb5kdc[22604](info): commencing operation
krb5kdc: starting...
W20260926 14:58:34.875465 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.004s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:58:34 dist-test-slave-f9xb krb5kdc[22604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:58:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:58:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:58:34.978057 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:42591
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:42591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:58:35.056110 22617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:35.056283 22617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:35.056304 22617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:35.057368 22617 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 14:58:35.057399 22617 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 14:58:35.057411 22617 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 14:58:35.057420 22617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:58:35.057432 22617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:35.057443 22617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:35.057453 22617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:58:35.057464 22617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:58:35.058816 22617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:42591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:42591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:35.058981 22617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:35.059128 22617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:35.063505 22622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:35.063511 22623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:35.063617 22617 server_base.cc:1034] running on GCE node
W20260926 14:58:35.063653 22625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:35.064002 22617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:35.064311 22617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:35.065464 22617 hybrid_clock.cc:648] HybridClock initialized: now 1790434715065448 us; error 29 us; skew 500 ppm
Sep 26 14:58:35 dist-test-slave-f9xb krb5kdc[22604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434715, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:35.067732 22617 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:58:35.068150 22617 webserver.cc:460] Webserver started at http://127.7.37.254:39927/ using document root <none> and password file <none>
I20260926 14:58:35.068358 22617 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:35.068404 22617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:35.068527 22617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:58:35.069476 22617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "42baa97c88214e84bad7f2f0421205a6"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "a531b386fb169ea4c3bb41fb5ee5ec4e"
server_key_iv: "b60a46bb38b6fa2fdbacf66a96da5a6a"
server_key_version: "encryptionkey@0"
I20260926 14:58:35.069799 22617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "42baa97c88214e84bad7f2f0421205a6"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "a531b386fb169ea4c3bb41fb5ee5ec4e"
server_key_iv: "b60a46bb38b6fa2fdbacf66a96da5a6a"
server_key_version: "encryptionkey@0"
I20260926 14:58:35.071046 22617 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 14:58:35.071691 22632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:35.071847 22617 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:35.071924 22617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "42baa97c88214e84bad7f2f0421205a6"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "a531b386fb169ea4c3bb41fb5ee5ec4e"
server_key_iv: "b60a46bb38b6fa2fdbacf66a96da5a6a"
server_key_version: "encryptionkey@0"
I20260926 14:58:35.071995 22617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:35.085665 22617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:35.087260 22617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:35.087378 22617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:35.091121 22617 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:42591
I20260926 14:58:35.091150 22684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:42591 every 8 connection(s)
I20260926 14:58:35.091530 22617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:58:35.092293 22685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:35.094187 22685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6: Bootstrap starting.
I20260926 14:58:35.094691 22685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:35.094931 22685 log.cc:824] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:35.095511 22685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6: No bootstrap required, opened a new log
I20260926 14:58:35.096966 22685 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42baa97c88214e84bad7f2f0421205a6" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42591 } }
I20260926 14:58:35.097087 22685 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:35.097107 22685 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42baa97c88214e84bad7f2f0421205a6, State: Initialized, Role: FOLLOWER
I20260926 14:58:35.097231 22685 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42baa97c88214e84bad7f2f0421205a6" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42591 } }
I20260926 14:58:35.097311 22685 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:58:35.097332 22685 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:58:35.097378 22685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:35.097826 22685 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42baa97c88214e84bad7f2f0421205a6" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42591 } }
I20260926 14:58:35.097908 22685 leader_election.cc:302] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42baa97c88214e84bad7f2f0421205a6; no voters:
I20260926 14:58:35.098052 22685 leader_election.cc:288] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:58:35.098096 22688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:58:35.098245 22688 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 1 LEADER]: Becoming Leader. State: Replica: 42baa97c88214e84bad7f2f0421205a6, State: Running, Role: LEADER
I20260926 14:58:35.098271 22685 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:58:35.098333 22688 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42baa97c88214e84bad7f2f0421205a6" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42591 } }
I20260926 14:58:35.098690 22690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42baa97c88214e84bad7f2f0421205a6. Latest consensus state: current_term: 1 leader_uuid: "42baa97c88214e84bad7f2f0421205a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42baa97c88214e84bad7f2f0421205a6" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42591 } } }
I20260926 14:58:35.098824 22690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [sys.catalog]: This master's current role is: LEADER
I20260926 14:58:35.098687 22689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42baa97c88214e84bad7f2f0421205a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42baa97c88214e84bad7f2f0421205a6" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42591 } } }
I20260926 14:58:35.098934 22689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [sys.catalog]: This master's current role is: LEADER
I20260926 14:58:35.099222 22697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:58:35.099697 22697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:58:35.101197 22697 catalog_manager.cc:1347] Generated new cluster ID: 1cb4114710e146e0aa7d4e0c2d3cefcc
I20260926 14:58:35.101244 22697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:58:35.101635 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 22617
I20260926 14:58:35.101733 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:58:35.101925 7319 external_mini_cluster.cc:1428] Setting key 8f1b99acd13cb48ee9916bd174cfc664
I20260926 14:58:35.107090 22697 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:58:35.107735 22697 catalog_manager.cc:1504] Loading token signing keys...
Sep 26 14:58:35 dist-test-slave-f9xb krb5kdc[22604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:35.113999 22708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.102437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34320 (local address 127.7.37.254:42591)
0926 14:58:35.103834 (+ 1397us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:35.103842 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:35.103871 (+ 29us) server_negotiation.cc:407] Connection header received
0926 14:58:35.103981 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:35.103986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:35.104036 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:35.104181 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:35.104863 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.106235 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.107291 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.107938 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.108050 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:35.110675 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:35.110679 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:35.110679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:35.110690 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:35.112502 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:35.112876 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:35.112878 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:35.112879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:35.112975 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:35.113158 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:35.113159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:35.113159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:35.113406 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:35.113537 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:35.113646 (+ 109us) server_negotiation.cc:299] Negotiation successful
0926 14:58:35.113789 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1359,"thread_start_us":71,"threads_started":1}
I20260926 14:58:35.114344 22697 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6: Generated new TSK 0
I20260926 14:58:35.114490 22697 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
I20260926 14:58:35.141732 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42591
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:58:35.217535 22711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:35.217717 22711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:35.217746 22711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:35.218842 22711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:35.218881 22711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:35.218916 22711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:58:35.220321 22711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:35.220533 22711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:35.220721 22711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:35.225255 22719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:35.225277 22716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:35.225363 22711 server_base.cc:1034] running on GCE node
W20260926 14:58:35.225323 22717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:35.225579 22711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:35.225773 22711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:35.226903 22711 hybrid_clock.cc:648] HybridClock initialized: now 1790434715226892 us; error 26 us; skew 500 ppm
Sep 26 14:58:35 dist-test-slave-f9xb krb5kdc[22604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434715, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:35.229102 22711 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:58:35.229446 22711 webserver.cc:460] Webserver started at http://127.7.37.193:39373/ using document root <none> and password file <none>
I20260926 14:58:35.229606 22711 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:35.229641 22711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:35.229730 22711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:58:35.230594 22711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "46d932a68cc240ea8d6667a10e36bb06"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "45349e80f48bda9539148412f18bc074"
server_key_iv: "09696fb6e6a916a91cdbe2e27c777dbe"
server_key_version: "encryptionkey@0"
I20260926 14:58:35.230870 22711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "46d932a68cc240ea8d6667a10e36bb06"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "45349e80f48bda9539148412f18bc074"
server_key_iv: "09696fb6e6a916a91cdbe2e27c777dbe"
server_key_version: "encryptionkey@0"
I20260926 14:58:35.231973 22711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:58:35.232556 22726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:35.232698 22711 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:35.232766 22711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "46d932a68cc240ea8d6667a10e36bb06"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "45349e80f48bda9539148412f18bc074"
server_key_iv: "09696fb6e6a916a91cdbe2e27c777dbe"
server_key_version: "encryptionkey@0"
I20260926 14:58:35.232824 22711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:35.240813 22711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:35.242259 22711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:35.242352 22711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:35.242497 22711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:35.242697 22711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:58:35.242722 22711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:35.242746 22711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:58:35.242763 22711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:35.247860 22711 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:45733
I20260926 14:58:35.247893 22839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:45733 every 8 connection(s)
I20260926 14:58:35.248168 22711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
Sep 26 14:58:35 dist-test-slave-f9xb krb5kdc[22604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434715, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:35.255539 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 22711
I20260926 14:58:35.255628 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:58:35.255822 7319 external_mini_cluster.cc:1428] Setting key 6f1eb4aadea1f0bf133eae38dba1ea5e
I20260926 14:58:35.256134 22708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.248734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:34379 (local address 127.7.37.254:42591)
0926 14:58:35.248782 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:35.248784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:35.248950 (+ 166us) server_negotiation.cc:407] Connection header received
0926 14:58:35.249365 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:35.249366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:35.249378 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:35.249423 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:35.250264 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.250740 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.251688 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.251931 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.251981 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:35.253770 (+ 1789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:35.253772 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:35.253772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:35.253779 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:35.254854 (+ 1075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:35.255181 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:35.255181 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:35.255182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:35.255253 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:35.255484 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:35.255485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:35.255485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:35.255600 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:35.255670 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:35.255964 (+ 294us) server_negotiation.cc:299] Negotiation successful
0926 14:58:35.256048 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 14:58:35.256271 22842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.248706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:34379)
0926 14:58:35.248852 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:35.248882 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:35.249283 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:35.249494 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:35.249498 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:35.250009 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:35.250171 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:35.250175 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.250817 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.250818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:35.251610 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:35.251612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.252007 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.252008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:35.252114 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:35.252501 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:35.252508 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:35.253681 (+ 1173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:35.254923 (+ 1242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:35.254930 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:35.254932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:35.255119 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:35.255309 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:35.255310 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:35.255310 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:35.255414 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:35.255673 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:35.255679 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:35.255840 (+ 161us) client_negotiation.cc:769] Sending connection context
0926 14:58:35.255943 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 14:58:35.256083 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260926 14:58:35.256726 22840 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42591
I20260926 14:58:35.256796 22840 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:35.256938 22840 heartbeater.cc:511] Master 127.7.37.254:42591 requested a full tablet report, sending...
I20260926 14:58:35.257417 22649 ts_manager.cc:194] Registered new tserver with Master: 46d932a68cc240ea8d6667a10e36bb06 (127.7.37.193:45733)
I20260926 14:58:35.258095 22649 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:34379
WARNING: no policy specified for kudu/127.7.37.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
I20260926 14:58:35.276059 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42591
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:58:35.350512 22845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:35.350679 22845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:35.350699 22845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:35.351804 22845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:35.351840 22845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:35.351866 22845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:58:35.353229 22845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:35.353423 22845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:35.353600 22845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:35.357877 22850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:35.357941 22845 server_base.cc:1034] running on GCE node
W20260926 14:58:35.357864 22851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:35.357862 22853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:35.358242 22845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:35.358471 22845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:35.359596 22845 hybrid_clock.cc:648] HybridClock initialized: now 1790434715359584 us; error 29 us; skew 500 ppm
Sep 26 14:58:35 dist-test-slave-f9xb krb5kdc[22604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434715, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:35.361783 22845 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:58:35.362138 22845 webserver.cc:460] Webserver started at http://127.7.37.194:45659/ using document root <none> and password file <none>
I20260926 14:58:35.362313 22845 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:35.362356 22845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:35.362468 22845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:58:35.363339 22845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "2342907514074c7fb83f34ca3dfafbe8"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "01f8074bb4b5e3f8764c3be08f04e081"
server_key_iv: "2b3f55e1bbdab8e0ae5519ec27cbdec8"
server_key_version: "encryptionkey@0"
I20260926 14:58:35.363628 22845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "2342907514074c7fb83f34ca3dfafbe8"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "01f8074bb4b5e3f8764c3be08f04e081"
server_key_iv: "2b3f55e1bbdab8e0ae5519ec27cbdec8"
server_key_version: "encryptionkey@0"
I20260926 14:58:35.364750 22845 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 14:58:35.365348 22860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:35.365483 22845 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:58:35.365552 22845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "2342907514074c7fb83f34ca3dfafbe8"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "01f8074bb4b5e3f8764c3be08f04e081"
server_key_iv: "2b3f55e1bbdab8e0ae5519ec27cbdec8"
server_key_version: "encryptionkey@0"
I20260926 14:58:35.365608 22845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:35.387370 22845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:35.388877 22845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:35.388994 22845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:35.389164 22845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:35.389405 22845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:58:35.389434 22845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:35.389458 22845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:58:35.389475 22845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:35.394562 22845 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:39837
I20260926 14:58:35.394619 22973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:39837 every 8 connection(s)
I20260926 14:58:35.394860 22845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:58:35.399560 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 22845
I20260926 14:58:35.399662 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:58:35.399842 7319 external_mini_cluster.cc:1428] Setting key 2bd22d619e9fc9d25c6611caa52ecaab
Sep 26 14:58:35 dist-test-slave-f9xb krb5kdc[22604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434715, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:35.403071 22708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.395450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:57027 (local address 127.7.37.254:42591)
0926 14:58:35.395499 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:35.395502 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:35.395651 (+ 149us) server_negotiation.cc:407] Connection header received
0926 14:58:35.396102 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:35.396103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:35.396120 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:35.396170 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:35.396969 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.397484 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.398419 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.398646 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.398700 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:35.400668 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:35.400670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:35.400670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:35.400678 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:35.401918 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:35.402339 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:35.402339 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:35.402339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:35.402383 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:35.402583 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:35.402584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:35.402584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:35.402669 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:35.402710 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:35.402962 (+ 252us) server_negotiation.cc:299] Negotiation successful
0926 14:58:35.403012 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 14:58:35.403285 22976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.395434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:57027)
0926 14:58:35.395557 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:35.395585 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:35.396016 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:35.396226 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:35.396230 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:35.396720 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:35.396882 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:35.396886 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.397561 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.397562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:35.398342 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:35.398344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.398713 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.398714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:35.398818 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:35.399205 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:35.399212 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:35.400595 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:35.401988 (+ 1393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:35.401994 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:35.401996 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:35.402263 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:35.402421 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:35.402421 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:35.402421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:35.402533 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:35.402715 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:35.402719 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:35.402853 (+ 134us) client_negotiation.cc:769] Sending connection context
0926 14:58:35.402950 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 14:58:35.403083 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":62,"threads_started":1}
I20260926 14:58:35.403698 22974 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42591
I20260926 14:58:35.403823 22974 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:35.403970 22974 heartbeater.cc:511] Master 127.7.37.254:42591 requested a full tablet report, sending...
I20260926 14:58:35.404269 22649 ts_manager.cc:194] Registered new tserver with Master: 2342907514074c7fb83f34ca3dfafbe8 (127.7.37.194:39837)
I20260926 14:58:35.404616 22649 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:57027
WARNING: no policy specified for kudu/127.7.37.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
I20260926 14:58:35.420151 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42591
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:58:35.494051 22979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:35.494211 22979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:35.494232 22979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:35.495338 22979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:35.495381 22979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:35.495411 22979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:58:35.496798 22979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:42591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:35.496984 22979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:35.497164 22979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:35.501541 22984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:35.501567 22985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:35.501659 22987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:35.501663 22979 server_base.cc:1034] running on GCE node
I20260926 14:58:35.501859 22979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:35.502041 22979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:35.503186 22979 hybrid_clock.cc:648] HybridClock initialized: now 1790434715503171 us; error 27 us; skew 500 ppm
Sep 26 14:58:35 dist-test-slave-f9xb krb5kdc[22604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434715, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:35.505273 22979 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:58:35.505651 22979 webserver.cc:460] Webserver started at http://127.7.37.195:45983/ using document root <none> and password file <none>
I20260926 14:58:35.505834 22979 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:35.505879 22979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:35.505990 22979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:58:35.506850 22979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "c383d5bbca814458b9ab7a2a62ad3321"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "445e874ad342de95903f1ecccc7e9658"
server_key_iv: "59209239c92b50d19579d14d15cbe76b"
server_key_version: "encryptionkey@0"
I20260926 14:58:35.507130 22979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "c383d5bbca814458b9ab7a2a62ad3321"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "445e874ad342de95903f1ecccc7e9658"
server_key_iv: "59209239c92b50d19579d14d15cbe76b"
server_key_version: "encryptionkey@0"
I20260926 14:58:35.508265 22979 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:58:35.508867 22994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:35.508999 22979 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:58:35.509071 22979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "c383d5bbca814458b9ab7a2a62ad3321"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "445e874ad342de95903f1ecccc7e9658"
server_key_iv: "59209239c92b50d19579d14d15cbe76b"
server_key_version: "encryptionkey@0"
I20260926 14:58:35.509124 22979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:35.520072 22979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:35.521510 22979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:35.521623 22979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:35.521783 22979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:35.522012 22979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:58:35.522042 22979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:35.522065 22979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:58:35.522089 22979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:35.527238 22979 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:40525
I20260926 14:58:35.527271 23107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:40525 every 8 connection(s)
I20260926 14:58:35.527510 22979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
Sep 26 14:58:35 dist-test-slave-f9xb krb5kdc[22604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434715, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:35.533599 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 22979
I20260926 14:58:35.533703 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:58:35.533855 7319 external_mini_cluster.cc:1428] Setting key 6e74ad60f968f4bfba1534e6e654bc72
I20260926 14:58:35.535197 22708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.528083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:47275 (local address 127.7.37.254:42591)
0926 14:58:35.528138 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:35.528140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:35.528275 (+ 135us) server_negotiation.cc:407] Connection header received
0926 14:58:35.528640 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:35.528641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:35.528650 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:35.528695 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:35.529428 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.529915 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.530762 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.530942 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.530980 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:35.532634 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:35.532636 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:35.532636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:35.532642 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:35.533993 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:35.534480 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:35.534481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:35.534482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:35.534542 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:35.534750 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:35.534751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:35.534751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:35.534851 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:35.534910 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:35.535081 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 14:58:35.535138 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 14:58:35.535367 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.528044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:47275)
0926 14:58:35.528173 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:35.528207 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:35.528578 (+ 371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:35.528738 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:35.528740 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:35.529202 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:35.529369 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:35.529373 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.529970 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.529970 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:35.530707 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:35.530709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.530987 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.530988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:35.531073 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:35.531401 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:35.531407 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:35.532560 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:35.534059 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:35.534064 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:35.534066 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:35.534417 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:35.534604 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:35.534605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:35.534605 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:35.534705 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:35.534903 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:35.534905 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:35.534999 (+ 94us) client_negotiation.cc:769] Sending connection context
0926 14:58:35.535066 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 14:58:35.535162 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":70,"threads_started":1}
I20260926 14:58:35.535735 23108 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42591
I20260926 14:58:35.535837 23108 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:35.535987 23108 heartbeater.cc:511] Master 127.7.37.254:42591 requested a full tablet report, sending...
I20260926 14:58:35.536262 22649 ts_manager.cc:194] Registered new tserver with Master: c383d5bbca814458b9ab7a2a62ad3321 (127.7.37.195:40525)
I20260926 14:58:35.536612 22649 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:47275
I20260926 14:58:35.545106 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 14:58:35.550956 22708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.546245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34332 (local address 127.7.37.254:42591)
0926 14:58:35.546276 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:35.546277 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:35.546385 (+ 108us) server_negotiation.cc:407] Connection header received
0926 14:58:35.546423 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:35.546423 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:35.546430 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:35.546467 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:35.546917 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.547397 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.548091 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.548267 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.548322 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:35.549199 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:35.549200 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:35.549200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:35.549206 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:35.550200 (+ 994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:35.550456 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:35.550456 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:35.550457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:35.550496 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:35.550610 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:35.550610 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:35.550610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:35.550714 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:35.550776 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:35.550855 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 14:58:35.550902 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 14:58:35.552263 22649 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34332:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:58:35.553632 22649 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:58:35.559760 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.555487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:39837 (local address 127.0.0.1:48098)
0926 14:58:35.555865 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:35.555874 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:35.555920 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:35.556114 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:35.556115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:35.556117 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:35.556142 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:35.556143 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.557319 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.557320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:35.558539 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:35.558541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.559494 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.559495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:35.559563 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:35.559575 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 14:58:35.559630 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 14:58:35.559674 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":58,"threads_started":1}
I20260926 14:58:35.560014 23121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.555671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48098 (local address 127.7.37.194:39837)
0926 14:58:35.555845 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:35.555849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:35.555968 (+ 119us) server_negotiation.cc:407] Connection header received
0926 14:58:35.556006 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:35.556009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:35.556023 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:35.556115 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:35.556185 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.557225 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.558613 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.559357 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.559417 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:35.559465 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:35.559902 (+ 437us) server_negotiation.cc:299] Negotiation successful
0926 14:58:35.559944 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260926 14:58:35.560621 23120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.555274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:40525 (local address 127.0.0.1:58188)
0926 14:58:35.555582 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:35.555590 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:35.555640 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:35.555909 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:35.555911 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:35.555913 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:35.555980 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:35.555982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.558430 (+ 2448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.558431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:35.559464 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:35.559467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.560429 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.560430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:35.560490 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:35.560493 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:58:35.560520 (+ 27us) client_negotiation.cc:240] Negotiation successful
0926 14:58:35.560556 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":4,"spinlock_wait_cycles":768,"thread_start_us":69,"threads_started":1}
I20260926 14:58:35.560829 22908 tablet_service.cc:1467] Processing CreateTablet for tablet 3e0651be61eb46a6ae2e7c21219293dc (DEFAULT_TABLE table=test-table [id=8c8f230d9a7c40848490c23db528cd83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:58:35.561000 23119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.555379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58188 (local address 127.7.37.195:40525)
0926 14:58:35.555606 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:35.555610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:35.555623 (+ 13us) server_negotiation.cc:407] Connection header received
0926 14:58:35.555710 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:35.555712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:35.555727 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:35.555901 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:35.557286 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.558324 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.559548 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.560345 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.560822 (+ 477us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:35.560881 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:35.560905 (+ 24us) server_negotiation.cc:299] Negotiation successful
0926 14:58:35.560937 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":78,"threads_started":1}
I20260926 14:58:35.561137 22908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e0651be61eb46a6ae2e7c21219293dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:35.561540 23042 tablet_service.cc:1467] Processing CreateTablet for tablet 3e0651be61eb46a6ae2e7c21219293dc (DEFAULT_TABLE table=test-table [id=8c8f230d9a7c40848490c23db528cd83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:58:35.561802 23042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e0651be61eb46a6ae2e7c21219293dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:35.562326 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.555274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:45733 (local address 127.0.0.1:40276)
0926 14:58:35.555541 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:35.555578 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:35.555640 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:35.555928 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:35.555929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:35.555930 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:35.555981 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:35.555982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.560472 (+ 4490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.560473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:35.561487 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:35.561490 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.562103 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.562104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:35.562161 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:35.562165 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:58:35.562217 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 14:58:35.562254 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":17,"thread_start_us":68,"threads_started":1}
I20260926 14:58:35.562342 23118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.555369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40276 (local address 127.7.37.193:45733)
0926 14:58:35.555538 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:35.555542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:35.555709 (+ 167us) server_negotiation.cc:407] Connection header received
0926 14:58:35.555768 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:35.555771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:35.555786 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:35.555854 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:35.556054 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.557180 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.561558 (+ 4378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.562034 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.562092 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:35.562142 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:35.562230 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 14:58:35.562269 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260926 14:58:35.562853 22774 tablet_service.cc:1467] Processing CreateTablet for tablet 3e0651be61eb46a6ae2e7c21219293dc (DEFAULT_TABLE table=test-table [id=8c8f230d9a7c40848490c23db528cd83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:58:35.563112 22774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e0651be61eb46a6ae2e7c21219293dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:35.564241 23122 tablet_bootstrap.cc:492] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8: Bootstrap starting.
I20260926 14:58:35.564726 23123 tablet_bootstrap.cc:492] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321: Bootstrap starting.
I20260926 14:58:35.564836 23122 tablet_bootstrap.cc:654] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:35.565272 23123 tablet_bootstrap.cc:654] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:35.565467 23122 log.cc:824] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:35.565732 23123 log.cc:824] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:35.566134 23122 tablet_bootstrap.cc:492] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8: No bootstrap required, opened a new log
I20260926 14:58:35.566184 23122 ts_tablet_manager.cc:1397] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:58:35.566378 23123 tablet_bootstrap.cc:492] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321: No bootstrap required, opened a new log
I20260926 14:58:35.566430 23123 ts_tablet_manager.cc:1397] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:58:35.567461 23122 raft_consensus.cc:348] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } }
I20260926 14:58:35.567582 23122 raft_consensus.cc:374] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:35.567601 23122 raft_consensus.cc:729] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2342907514074c7fb83f34ca3dfafbe8, State: Initialized, Role: FOLLOWER
I20260926 14:58:35.567705 23122 consensus_queue.cc:260] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } }
I20260926 14:58:35.567731 23123 raft_consensus.cc:348] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } }
I20260926 14:58:35.567953 23123 raft_consensus.cc:374] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:35.567982 23122 ts_tablet_manager.cc:1428] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:58:35.568029 23125 tablet_bootstrap.cc:492] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06: Bootstrap starting.
I20260926 14:58:35.568096 22974 heartbeater.cc:503] Master 127.7.37.254:42591 was elected leader, sending a full tablet report...
I20260926 14:58:35.568113 23123 raft_consensus.cc:729] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c383d5bbca814458b9ab7a2a62ad3321, State: Initialized, Role: FOLLOWER
I20260926 14:58:35.568259 23123 consensus_queue.cc:260] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } }
I20260926 14:58:35.568513 23123 ts_tablet_manager.cc:1428] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 14:58:35.568586 23125 tablet_bootstrap.cc:654] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:35.568614 23108 heartbeater.cc:503] Master 127.7.37.254:42591 was elected leader, sending a full tablet report...
I20260926 14:58:35.569110 23125 log.cc:824] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:35.569722 23125 tablet_bootstrap.cc:492] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06: No bootstrap required, opened a new log
I20260926 14:58:35.569772 23125 ts_tablet_manager.cc:1397] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260926 14:58:35.570986 23125 raft_consensus.cc:348] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } }
I20260926 14:58:35.571101 23125 raft_consensus.cc:374] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:35.571131 23125 raft_consensus.cc:729] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46d932a68cc240ea8d6667a10e36bb06, State: Initialized, Role: FOLLOWER
I20260926 14:58:35.571230 23125 consensus_queue.cc:260] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } }
I20260926 14:58:35.571466 23125 ts_tablet_manager.cc:1428] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:58:35.571499 22840 heartbeater.cc:503] Master 127.7.37.254:42591 was elected leader, sending a full tablet report...
W20260926 14:58:35.645335 22975 tablet.cc:2369] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 14:58:35.748827 22841 tablet.cc:2369] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 14:58:35.778005 23109 tablet.cc:2369] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:58:35.823012 23128 raft_consensus.cc:482] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:58:35.823120 23128 raft_consensus.cc:504] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } }
I20260926 14:58:35.823438 23128 leader_election.cc:288] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2342907514074c7fb83f34ca3dfafbe8 (127.7.37.194:39837), 46d932a68cc240ea8d6667a10e36bb06 (127.7.37.193:45733)
I20260926 14:58:35.827081 23121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.823693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:59403 (local address 127.7.37.194:39837)
0926 14:58:35.823766 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:35.823768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:35.823778 (+ 10us) server_negotiation.cc:407] Connection header received
0926 14:58:35.823804 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:35.823805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:35.823823 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:35.823868 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:35.824046 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.824777 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.826185 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.826735 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.826794 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:35.826801 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:35.826978 (+ 177us) server_negotiation.cc:299] Negotiation successful
0926 14:58:35.827015 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 14:58:35.827081 23118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.823664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:36637 (local address 127.7.37.193:45733)
0926 14:58:35.823909 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:35.823911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:35.823916 (+ 5us) server_negotiation.cc:407] Connection header received
0926 14:58:35.823935 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:35.823936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:35.823943 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:35.823978 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:35.824131 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.824802 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.826184 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.826720 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.826793 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:35.826801 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:35.826981 (+ 180us) server_negotiation.cc:299] Negotiation successful
0926 14:58:35.827015 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 14:58:35.827095 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.823573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:39837 (local address 127.7.37.195:59403)
0926 14:58:35.823623 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:35.823642 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:35.823697 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:35.823892 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:35.823893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:35.823894 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:35.823967 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:35.823969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.824858 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.824859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:35.826101 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:35.826104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.826800 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.826801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:35.826905 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:35.826907 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:58:35.826967 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 14:58:35.827003 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":21}
I20260926 14:58:35.827101 23131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.823581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:45733 (local address 127.7.37.195:36637)
0926 14:58:35.823799 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:35.823808 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:35.823862 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:35.823995 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:35.823996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:35.823997 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:35.824069 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:35.824071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.824877 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.824878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:35.826101 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:35.826104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.826801 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.826801 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:35.826902 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:35.826907 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 14:58:35.826961 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 14:58:35.827002 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":86,"thread_start_us":78,"threads_started":1}
I20260926 14:58:35.827431 22928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e0651be61eb46a6ae2e7c21219293dc" candidate_uuid: "c383d5bbca814458b9ab7a2a62ad3321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2342907514074c7fb83f34ca3dfafbe8" is_pre_election: true
I20260926 14:58:35.827595 22928 raft_consensus.cc:2454] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c383d5bbca814458b9ab7a2a62ad3321 in term 0.
I20260926 14:58:35.827627 22794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e0651be61eb46a6ae2e7c21219293dc" candidate_uuid: "c383d5bbca814458b9ab7a2a62ad3321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46d932a68cc240ea8d6667a10e36bb06" is_pre_election: true
I20260926 14:58:35.827787 22794 raft_consensus.cc:2454] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c383d5bbca814458b9ab7a2a62ad3321 in term 0.
I20260926 14:58:35.827841 22997 leader_election.cc:302] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2342907514074c7fb83f34ca3dfafbe8, c383d5bbca814458b9ab7a2a62ad3321; no voters:
I20260926 14:58:35.827962 23128 raft_consensus.cc:2781] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:58:35.828025 23128 raft_consensus.cc:482] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:58:35.828040 23128 raft_consensus.cc:3037] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:35.828706 23128 raft_consensus.cc:504] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } }
I20260926 14:58:35.828811 23128 leader_election.cc:288] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [CANDIDATE]: Term 1 election: Requested vote from peers 2342907514074c7fb83f34ca3dfafbe8 (127.7.37.194:39837), 46d932a68cc240ea8d6667a10e36bb06 (127.7.37.193:45733)
I20260926 14:58:35.828965 22794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e0651be61eb46a6ae2e7c21219293dc" candidate_uuid: "c383d5bbca814458b9ab7a2a62ad3321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46d932a68cc240ea8d6667a10e36bb06"
I20260926 14:58:35.828979 22928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e0651be61eb46a6ae2e7c21219293dc" candidate_uuid: "c383d5bbca814458b9ab7a2a62ad3321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2342907514074c7fb83f34ca3dfafbe8"
I20260926 14:58:35.829038 22794 raft_consensus.cc:3037] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:35.829038 22928 raft_consensus.cc:3037] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:35.829828 22928 raft_consensus.cc:2454] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c383d5bbca814458b9ab7a2a62ad3321 in term 1.
I20260926 14:58:35.829833 22794 raft_consensus.cc:2454] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c383d5bbca814458b9ab7a2a62ad3321 in term 1.
I20260926 14:58:35.830050 22997 leader_election.cc:302] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2342907514074c7fb83f34ca3dfafbe8, c383d5bbca814458b9ab7a2a62ad3321; no voters:
I20260926 14:58:35.830139 23128 raft_consensus.cc:2781] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:58:35.830247 23128 raft_consensus.cc:686] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [term 1 LEADER]: Becoming Leader. State: Replica: c383d5bbca814458b9ab7a2a62ad3321, State: Running, Role: LEADER
I20260926 14:58:35.830317 23128 consensus_queue.cc:237] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } }
I20260926 14:58:35.831019 22648 catalog_manager.cc:5495] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 reported cstate change: term changed from 0 to 1, leader changed from <none> to c383d5bbca814458b9ab7a2a62ad3321 (127.7.37.195). New cstate: current_term: 1 leader_uuid: "c383d5bbca814458b9ab7a2a62ad3321" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } health_report { overall_health: UNKNOWN } } }
I20260926 14:58:35.842356 23119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.840527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58194 (local address 127.7.37.195:40525)
0926 14:58:35.840569 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:35.840571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:35.840580 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:58:35.840608 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:35.840608 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:35.840621 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:35.840673 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:58:35.840767 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.841185 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.841830 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:35.841994 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:35.842033 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:35.842131 (+ 98us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:58:35.842211 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:58:35.842260 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:35.842275 (+ 15us) server_negotiation.cc:299] Negotiation successful
0926 14:58:35.842288 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 14:58:35.843735 22928 raft_consensus.cc:1261] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8 [term 1 FOLLOWER]: Refusing update from remote peer c383d5bbca814458b9ab7a2a62ad3321: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:58:35.843739 22794 raft_consensus.cc:1261] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06 [term 1 FOLLOWER]: Refusing update from remote peer c383d5bbca814458b9ab7a2a62ad3321: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:58:35.844060 23128 consensus_queue.cc:1035] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:58:35.844183 23132 consensus_queue.cc:1035] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:58:35.846431 23136 mvcc.cc:205] Tried to move back new op lower bound from 7333620596093964288 to 7333620596041449472. Current Snapshot: MvccSnapshot[applied={T|T < 7333620596093964288}]
I20260926 14:58:35.847095 23137 mvcc.cc:205] Tried to move back new op lower bound from 7333620596093964288 to 7333620596041449472. Current Snapshot: MvccSnapshot[applied={T|T < 7333620596093964288}]
I20260926 14:58:35.848142 22649 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:58:35.848213 22649 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:58:35.848839 22649 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6: Sending DeleteTablet for 3 replicas of tablet 3e0651be61eb46a6ae2e7c21219293dc
I20260926 14:58:35.849123 22774 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e0651be61eb46a6ae2e7c21219293dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:58:35 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:40276
I20260926 14:58:35.849128 23042 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e0651be61eb46a6ae2e7c21219293dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:58:35 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:58188
I20260926 14:58:35.849306 22908 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e0651be61eb46a6ae2e7c21219293dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:58:35 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:48098
I20260926 14:58:35.849329 23144 tablet_replica.cc:330] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06: stopping tablet replica
I20260926 14:58:35.849411 23144 raft_consensus.cc:2229] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:58:35.849413 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 22711
I20260926 14:58:35.849504 23144 raft_consensus.cc:2258] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:58:35.849583 23145 tablet_replica.cc:330] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321: stopping tablet replica
I20260926 14:58:35.849717 23145 raft_consensus.cc:2229] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:58:35.849848 23145 raft_consensus.cc:2258] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:58:35.850282 23145 ts_tablet_manager.cc:1905] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:58:35.851998 23145 ts_tablet_manager.cc:1918] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:58:35.852100 23145 log.cc:1196] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/3e0651be61eb46a6ae2e7c21219293dc
I20260926 14:58:35.852377 23145 ts_tablet_manager.cc:1939] T 3e0651be61eb46a6ae2e7c21219293dc P c383d5bbca814458b9ab7a2a62ad3321: Deleting consensus metadata
I20260926 14:58:35.852746 22635 catalog_manager.cc:4841] TS c383d5bbca814458b9ab7a2a62ad3321 (127.7.37.195:40525): tablet 3e0651be61eb46a6ae2e7c21219293dc (table test-table [id=8c8f230d9a7c40848490c23db528cd83]) successfully deleted
W20260926 14:58:35.854192 22634 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.37.193:45733 (error 108)
I20260926 14:58:35.854408 23146 tablet_replica.cc:330] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8: stopping tablet replica
I20260926 14:58:35.854501 23146 raft_consensus.cc:2229] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:58:35.854589 23146 raft_consensus.cc:2258] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:58:35.854637 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.854527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:45733 (local address 127.0.0.1:40292)
0926 14:58:35.854573 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:35.854607 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.193:45733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 14:58:35.854744 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 22845
W20260926 14:58:35.854830 22634 catalog_manager.cc:4584] TS 46d932a68cc240ea8d6667a10e36bb06 (127.7.37.193:45733): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e0651be61eb46a6ae2e7c21219293dc: Network error: Client connection negotiation failed: client connection to 127.7.37.193:45733: connect: Connection refused (error 111)
I20260926 14:58:35.858237 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:35.858152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:39837 (local address 127.0.0.1:48100)
0926 14:58:35.858194 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:35.858215 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.194:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260926 14:58:35.858762 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 22979
I20260926 14:58:35.862668 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 22617
I20260926 14:58:55.866789 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:42591
--webserver_interface=127.7.37.254
--webserver_port=39927
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:42591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:58:55.941793 23148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:55.941982 23148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:55.942004 23148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:55.943127 23148 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 14:58:55.943171 23148 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 14:58:55.943185 23148 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 14:58:55.943197 23148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:58:55.943214 23148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:55.943244 23148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:55.943270 23148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:58:55.943296 23148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:58:55.944739 23148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:42591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:42591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=39927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:55.944954 23148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:55.945158 23148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:55.952832 23157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:55.952898 23148 server_base.cc:1034] running on GCE node
W20260926 14:58:55.952827 23155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:55.952826 23154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:55.953269 23148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:55.953563 23148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:55.954725 23148 hybrid_clock.cc:648] HybridClock initialized: now 1790434735954712 us; error 23 us; skew 500 ppm
Sep 26 14:58:55 dist-test-slave-f9xb krb5kdc[22604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434735, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:55.963797 23148 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:58:55.964320 23148 webserver.cc:460] Webserver started at http://127.7.37.254:39927/ using document root <none> and password file <none>
I20260926 14:58:55.964558 23148 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:55.964617 23148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:55.966181 23148 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:58:55.966815 23164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:55.966967 23148 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:58:55.967052 23148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "42baa97c88214e84bad7f2f0421205a6"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "a531b386fb169ea4c3bb41fb5ee5ec4e"
server_key_iv: "b60a46bb38b6fa2fdbacf66a96da5a6a"
server_key_version: "encryptionkey@0"
I20260926 14:58:55.967453 23148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:55.985803 23148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:55.987480 23148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:55.987584 23148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:55.991096 23148 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:42591
I20260926 14:58:55.991145 23216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:42591 every 8 connection(s)
I20260926 14:58:55.991475 23148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:58:55.992753 23217 sys_catalog.cc:263] Verifying existing consensus state
I20260926 14:58:55.993345 23217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6: Bootstrap starting.
I20260926 14:58:56.000794 23217 log.cc:824] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:56.001327 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 23148
I20260926 14:58:56.001607 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:45733
--local_ip_for_outbound_sockets=127.7.37.193
--tserver_master_addrs=127.7.37.254:42591
--webserver_port=39373
--webserver_interface=127.7.37.193
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 14:58:56.002935 23217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 14:58:56.003180 23217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6: Bootstrap complete.
I20260926 14:58:56.005897 23217 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42baa97c88214e84bad7f2f0421205a6" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42591 } }
I20260926 14:58:56.006192 23217 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42baa97c88214e84bad7f2f0421205a6, State: Initialized, Role: FOLLOWER
I20260926 14:58:56.006343 23217 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42baa97c88214e84bad7f2f0421205a6" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42591 } }
I20260926 14:58:56.006417 23217 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:58:56.006465 23217 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:58:56.006513 23217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 1 FOLLOWER]: Advancing to term 2
I20260926 14:58:56.007802 23217 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42baa97c88214e84bad7f2f0421205a6" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42591 } }
I20260926 14:58:56.007908 23217 leader_election.cc:302] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42baa97c88214e84bad7f2f0421205a6; no voters:
I20260926 14:58:56.008054 23217 leader_election.cc:288] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 14:58:56.008108 23221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 14:58:56.008276 23217 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:58:56.008307 23221 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 2 LEADER]: Becoming Leader. State: Replica: 42baa97c88214e84bad7f2f0421205a6, State: Running, Role: LEADER
I20260926 14:58:56.008452 23221 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42baa97c88214e84bad7f2f0421205a6" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42591 } }
I20260926 14:58:56.008742 23222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "42baa97c88214e84bad7f2f0421205a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42baa97c88214e84bad7f2f0421205a6" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42591 } } }
I20260926 14:58:56.008819 23222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [sys.catalog]: This master's current role is: LEADER
I20260926 14:58:56.009120 23223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42baa97c88214e84bad7f2f0421205a6. Latest consensus state: current_term: 2 leader_uuid: "42baa97c88214e84bad7f2f0421205a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42baa97c88214e84bad7f2f0421205a6" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42591 } } }
I20260926 14:58:56.009214 23223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [sys.catalog]: This master's current role is: LEADER
I20260926 14:58:56.009361 23228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:58:56.010118 23228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:58:56.010269 23228 catalog_manager.cc:1259] Loaded cluster ID: 1cb4114710e146e0aa7d4e0c2d3cefcc
I20260926 14:58:56.010486 23228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:58:56.011667 23228 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:58:56.011881 23228 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6: Loaded TSK: 0
I20260926 14:58:56.022148 23228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6: Generated new TSK 1
I20260926 14:58:56.022225 23228 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 14:58:56.079265 23220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:56.079428 23220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:56.079449 23220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:56.080595 23220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:56.080634 23220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:56.080658 23220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:58:56.082041 23220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:45733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=39373
--tserver_master_addrs=127.7.37.254:42591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:56.082242 23220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:56.082438 23220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:56.087191 23244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:56.087270 23245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:56.087365 23220 server_base.cc:1034] running on GCE node
W20260926 14:58:56.087388 23247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:56.087551 23220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:56.087795 23220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:56.088934 23220 hybrid_clock.cc:648] HybridClock initialized: now 1790434736088923 us; error 29 us; skew 500 ppm
Sep 26 14:58:56 dist-test-slave-f9xb krb5kdc[22604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434736, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:56.091174 23220 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:58:56.091583 23220 webserver.cc:460] Webserver started at http://127.7.37.193:39373/ using document root <none> and password file <none>
I20260926 14:58:56.091828 23220 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:56.091890 23220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:56.093144 23220 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 14:58:56.093730 23254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:56.093878 23220 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:56.093951 23220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "46d932a68cc240ea8d6667a10e36bb06"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "45349e80f48bda9539148412f18bc074"
server_key_iv: "09696fb6e6a916a91cdbe2e27c777dbe"
server_key_version: "encryptionkey@0"
I20260926 14:58:56.094265 23220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:56.102210 23220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:56.104063 23220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:56.104176 23220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:56.104344 23220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:56.104694 23261 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 14:58:56.106459 23220 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 14:58:56.106498 23220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 14:58:56.106527 23220 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 14:58:56.107069 23220 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 14:58:56.107098 23220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 14:58:56.107193 23261 tablet_bootstrap.cc:492] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06: Bootstrap starting.
I20260926 14:58:56.113337 23220 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:45733
I20260926 14:58:56.113646 23220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:58:56.116273 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 23220
I20260926 14:58:56.116535 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:39837
--local_ip_for_outbound_sockets=127.7.37.194
--tserver_master_addrs=127.7.37.254:42591
--webserver_port=45659
--webserver_interface=127.7.37.194
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 14:58:56.117592 23261 log.cc:824] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:56.118465 23368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:45733 every 8 connection(s)
I20260926 14:58:56.120887 23261 tablet_bootstrap.cc:492] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 14:58:56.121163 23261 tablet_bootstrap.cc:492] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06: Bootstrap complete.
I20260926 14:58:56.121951 23261 ts_tablet_manager.cc:1397] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.008s
Sep 26 14:58:56 dist-test-slave-f9xb krb5kdc[22604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434736, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:56.122963 23261 raft_consensus.cc:348] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } }
I20260926 14:58:56.123243 23261 raft_consensus.cc:729] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46d932a68cc240ea8d6667a10e36bb06, State: Initialized, Role: FOLLOWER
I20260926 14:58:56.123391 23261 consensus_queue.cc:260] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } }
I20260926 14:58:56.123649 23261 ts_tablet_manager.cc:1428] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260926 14:58:56.126128 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.114188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:42537)
0926 14:58:56.114353 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:56.114390 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:56.114784 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:56.116300 (+ 1516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:56.116304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:56.116621 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:56.116875 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:56.116881 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.118343 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.118344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:56.119297 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:56.119300 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.120141 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.120143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:56.120305 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.120816 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:56.120825 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:56.122477 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:56.124550 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:56.124554 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:56.124556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:56.124746 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:56.125004 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:56.125005 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:56.125005 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:56.125134 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:56.125559 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:56.125564 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:56.125701 (+ 137us) client_negotiation.cc:769] Sending connection context
0926 14:58:56.125798 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 14:58:56.125944 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260926 14:58:56.126201 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.114267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:42537 (local address 127.7.37.254:42591)
0926 14:58:56.115883 (+ 1616us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:56.115890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:56.115914 (+ 24us) server_negotiation.cc:407] Connection header received
0926 14:58:56.116008 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:56.116013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:56.116051 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:56.116165 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:56.117022 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.118223 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.119388 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.120019 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.120144 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.122579 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:56.122583 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:56.122583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:56.122594 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:56.124479 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:56.124821 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:56.124825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:56.124825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:56.124932 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:56.125199 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:56.125200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:56.125200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:56.125476 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:56.125553 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:56.125866 (+ 313us) server_negotiation.cc:299] Negotiation successful
0926 14:58:56.125996 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1572,"thread_start_us":76,"threads_started":1}
I20260926 14:58:56.127151 23369 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42591
I20260926 14:58:56.127276 23369 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:56.127458 23369 heartbeater.cc:511] Master 127.7.37.254:42591 requested a full tablet report, sending...
I20260926 14:58:56.128211 23181 ts_manager.cc:194] Registered new tserver with Master: 46d932a68cc240ea8d6667a10e36bb06 (127.7.37.193:45733)
I20260926 14:58:56.129326 23181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:42537
I20260926 14:58:56.130244 23369 heartbeater.cc:503] Master 127.7.37.254:42591 was elected leader, sending a full tablet report...
Sep 26 14:58:56 dist-test-slave-f9xb krb5kdc[22604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434735, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for kudu/127.7.37.193@KRBTEST.COM
I20260926 14:58:56.137480 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.128768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:45733 (local address 127.0.0.1:33716)
0926 14:58:56.129217 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:56.129254 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:56.129319 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:56.129556 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:56.129561 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:56.129884 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:56.129939 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:56.129941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.131124 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.131125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:56.132013 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:56.132016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.132598 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.132598 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:56.132665 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.133083 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:56.133087 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:56.134577 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:56.136355 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:56.136361 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:56.136363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:56.136602 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:56.136780 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:56.136780 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:56.136781 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:56.136844 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:56.137131 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:56.137137 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:56.137188 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 14:58:56.137270 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 14:58:56.137369 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":82,"threads_started":1}
I20260926 14:58:56.137483 23377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.128990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33716 (local address 127.7.37.193:45733)
0926 14:58:56.129152 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:56.129156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:56.129392 (+ 236us) server_negotiation.cc:407] Connection header received
0926 14:58:56.129425 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:56.129428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:56.129441 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:56.129496 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:56.130028 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.131040 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.132086 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.132520 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.132583 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.134656 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:56.134663 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:56.134663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:56.134670 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:56.136270 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:56.136668 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:56.136670 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:56.136671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:56.136722 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:56.136902 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:56.136902 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:56.136902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:56.137049 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:56.137116 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:56.137308 (+ 192us) server_negotiation.cc:299] Negotiation successful
0926 14:58:56.137395 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":64,"threads_started":1}
I20260926 14:58:56.138370 23295 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e0651be61eb46a6ae2e7c21219293dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:58:35 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:33716
I20260926 14:58:56.138814 23378 tablet_replica.cc:330] stopping tablet replica
I20260926 14:58:56.138909 23378 raft_consensus.cc:2229] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:58:56.138964 23378 raft_consensus.cc:2258] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:58:56.139213 23378 ts_tablet_manager.cc:1905] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:58:56.141503 23378 ts_tablet_manager.cc:1918] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:58:56.141671 23378 log.cc:1196] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/wal/wals/3e0651be61eb46a6ae2e7c21219293dc
I20260926 14:58:56.142072 23378 ts_tablet_manager.cc:1939] T 3e0651be61eb46a6ae2e7c21219293dc P 46d932a68cc240ea8d6667a10e36bb06: Deleting consensus metadata
I20260926 14:58:56.142580 23166 catalog_manager.cc:4841] TS 46d932a68cc240ea8d6667a10e36bb06 (127.7.37.193:45733): tablet 3e0651be61eb46a6ae2e7c21219293dc (table test-table [id=8c8f230d9a7c40848490c23db528cd83]) successfully deleted
W20260926 14:58:56.201298 23373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:56.201464 23373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:56.201484 23373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:56.202512 23373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:56.202541 23373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:56.202565 23373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:58:56.203879 23373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:39837
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=45659
--tserver_master_addrs=127.7.37.254:42591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:56.204041 23373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:56.204231 23373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:56.208498 23384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:56.208505 23383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:56.208671 23373 server_base.cc:1034] running on GCE node
W20260926 14:58:56.208714 23386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:56.208875 23373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:56.209077 23373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:56.210230 23373 hybrid_clock.cc:648] HybridClock initialized: now 1790434736210217 us; error 24 us; skew 500 ppm
Sep 26 14:58:56 dist-test-slave-f9xb krb5kdc[22604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434736, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:56.212365 23373 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:58:56.212716 23373 webserver.cc:460] Webserver started at http://127.7.37.194:45659/ using document root <none> and password file <none>
I20260926 14:58:56.212885 23373 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:56.212941 23373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:56.214083 23373 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:58:56.214603 23393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:56.214748 23373 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:58:56.214821 23373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "2342907514074c7fb83f34ca3dfafbe8"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "01f8074bb4b5e3f8764c3be08f04e081"
server_key_iv: "2b3f55e1bbdab8e0ae5519ec27cbdec8"
server_key_version: "encryptionkey@0"
I20260926 14:58:56.215076 23373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:56.223105 23373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:56.224642 23373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:56.224754 23373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:56.224920 23373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:56.225246 23400 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 14:58:56.226450 23373 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 14:58:56.226511 23373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260926 14:58:56.226544 23373 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 14:58:56.227118 23373 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 14:58:56.227154 23373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 14:58:56.227214 23400 tablet_bootstrap.cc:492] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8: Bootstrap starting.
I20260926 14:58:56.232983 23373 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:39837
I20260926 14:58:56.233053 23507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:39837 every 8 connection(s)
I20260926 14:58:56.233327 23373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:58:56.236382 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 23373
I20260926 14:58:56.236564 23400 log.cc:824] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:56.236678 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:40525
--local_ip_for_outbound_sockets=127.7.37.195
--tserver_master_addrs=127.7.37.254:42591
--webserver_port=45983
--webserver_interface=127.7.37.195
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 14:58:56.238871 23400 tablet_bootstrap.cc:492] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 14:58:56.239144 23400 tablet_bootstrap.cc:492] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8: Bootstrap complete.
Sep 26 14:58:56 dist-test-slave-f9xb krb5kdc[22604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434736, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:56.239866 23400 ts_tablet_manager.cc:1397] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.012s
I20260926 14:58:56.240869 23400 raft_consensus.cc:348] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } }
I20260926 14:58:56.241107 23400 raft_consensus.cc:729] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2342907514074c7fb83f34ca3dfafbe8, State: Initialized, Role: FOLLOWER
I20260926 14:58:56.241256 23400 consensus_queue.cc:260] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } }
I20260926 14:58:56.241513 23400 ts_tablet_manager.cc:1428] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 14:58:56.243871 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.234031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:50729 (local address 127.7.37.254:42591)
0926 14:58:56.234096 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:56.234099 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:56.234326 (+ 227us) server_negotiation.cc:407] Connection header received
0926 14:58:56.234914 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:56.234915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:56.234929 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:56.234988 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:56.235842 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.236482 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.237569 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.237780 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.237833 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.240315 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:56.240317 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:56.240317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:56.240325 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:56.242564 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:56.242939 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:56.242940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:56.242941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:56.243000 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:56.243248 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:56.243249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:56.243249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:56.243380 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:56.243449 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:56.243663 (+ 214us) server_negotiation.cc:299] Negotiation successful
0926 14:58:56.243780 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 14:58:56.243928 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.233943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:50729)
0926 14:58:56.234173 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:56.234226 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:56.234809 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:56.235071 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:56.235076 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:56.235434 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:56.235708 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:56.235717 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.236578 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.236579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:56.237489 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:56.237491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.237860 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.237862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:56.237995 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.238576 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:56.238591 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:56.240206 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:56.242653 (+ 2447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:56.242659 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:56.242660 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:56.242873 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:56.243093 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:56.243094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:56.243094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:56.243186 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:56.243441 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:56.243444 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:56.243540 (+ 96us) client_negotiation.cc:769] Sending connection context
0926 14:58:56.243620 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 14:58:56.243776 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":102,"threads_started":1}
I20260926 14:58:56.244480 23508 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42591
I20260926 14:58:56.244568 23508 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:56.244736 23508 heartbeater.cc:511] Master 127.7.37.254:42591 requested a full tablet report, sending...
I20260926 14:58:56.245244 23181 ts_manager.cc:194] Registered new tserver with Master: 2342907514074c7fb83f34ca3dfafbe8 (127.7.37.194:39837)
I20260926 14:58:56.245828 23181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:50729
I20260926 14:58:56.246563 23508 heartbeater.cc:503] Master 127.7.37.254:42591 was elected leader, sending a full tablet report...
I20260926 14:58:56.250005 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.245519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:39837 (local address 127.0.0.1:39282)
0926 14:58:56.245563 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:56.245572 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:56.245607 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:56.247013 (+ 1406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:56.247014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:56.247015 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:56.247056 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:56.247057 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.248246 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.248247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:56.248988 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:56.248991 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.249815 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.249815 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:56.249870 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.249886 (+ 16us) client_negotiation.cc:769] Sending connection context
0926 14:58:56.249927 (+ 41us) client_negotiation.cc:240] Negotiation successful
0926 14:58:56.249958 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 14:58:56.250027 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.245566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39282 (local address 127.7.37.194:39837)
0926 14:58:56.246046 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:56.246750 (+ 704us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:56.246761 (+ 11us) server_negotiation.cc:407] Connection header received
0926 14:58:56.246905 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:56.246907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:56.246923 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:56.246998 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:56.247107 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.248155 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.249058 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.249736 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.249815 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.249863 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:56.249938 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 14:58:56.249968 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":64,"threads_started":1}
I20260926 14:58:56.250360 23442 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e0651be61eb46a6ae2e7c21219293dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:58:35 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:39282
I20260926 14:58:56.250787 23515 tablet_replica.cc:330] stopping tablet replica
I20260926 14:58:56.250866 23515 raft_consensus.cc:2229] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:58:56.250927 23515 raft_consensus.cc:2258] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:58:56.251163 23515 ts_tablet_manager.cc:1905] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:58:56.253144 23515 ts_tablet_manager.cc:1918] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:58:56.253206 23515 log.cc:1196] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/3e0651be61eb46a6ae2e7c21219293dc
I20260926 14:58:56.253388 23515 ts_tablet_manager.cc:1939] T 3e0651be61eb46a6ae2e7c21219293dc P 2342907514074c7fb83f34ca3dfafbe8: Deleting consensus metadata
I20260926 14:58:56.253787 23167 catalog_manager.cc:4841] TS 2342907514074c7fb83f34ca3dfafbe8 (127.7.37.194:39837): tablet 3e0651be61eb46a6ae2e7c21219293dc (table test-table [id=8c8f230d9a7c40848490c23db528cd83]) successfully deleted
W20260926 14:58:56.315562 23512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:58:56.315728 23512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:58:56.315775 23512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:58:56.316867 23512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:58:56.316910 23512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:58:56.316942 23512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:58:56.318400 23512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:40525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=45983
--tserver_master_addrs=127.7.37.254:42591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:58:56.318602 23512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:58:56.318785 23512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:58:56.323112 23523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:56.323112 23520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:58:56.323132 23521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:58:56.323272 23512 server_base.cc:1034] running on GCE node
I20260926 14:58:56.323530 23512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:58:56.323758 23512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:58:56.324927 23512 hybrid_clock.cc:648] HybridClock initialized: now 1790434736324915 us; error 29 us; skew 500 ppm
Sep 26 14:58:56 dist-test-slave-f9xb krb5kdc[22604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434736, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:58:56.327257 23512 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:58:56.327648 23512 webserver.cc:460] Webserver started at http://127.7.37.195:45983/ using document root <none> and password file <none>
I20260926 14:58:56.327880 23512 fs_manager.cc:359] Metadata directory not provided
I20260926 14:58:56.327944 23512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:58:56.329128 23512 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:58:56.329620 23530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:56.329768 23512 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:58:56.329856 23512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "c383d5bbca814458b9ab7a2a62ad3321"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "445e874ad342de95903f1ecccc7e9658"
server_key_iv: "59209239c92b50d19579d14d15cbe76b"
server_key_version: "encryptionkey@0"
I20260926 14:58:56.330123 23512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:58:56.366837 23512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:58:56.368892 23512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:58:56.369020 23512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:58:56.369228 23512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:58:56.369504 23512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:58:56.369539 23512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:56.369578 23512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:58:56.369606 23512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:56.375267 23512 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:40525
I20260926 14:58:56.375301 23643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:40525 every 8 connection(s)
I20260926 14:58:56.375602 23512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
Sep 26 14:58:56 dist-test-slave-f9xb krb5kdc[22604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434736, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:58:56.381847 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 23512
I20260926 14:58:56.383854 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.376213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:44445 (local address 127.7.37.254:42591)
0926 14:58:56.376259 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:56.376261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:56.376438 (+ 177us) server_negotiation.cc:407] Connection header received
0926 14:58:56.376848 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:56.376849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:56.376859 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:56.376914 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:56.377765 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.378246 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.379135 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.379324 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.379377 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.381178 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:56.381179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:56.381179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:56.381187 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:56.382648 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:56.382975 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:56.382976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:56.382976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:56.383033 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:56.383287 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:56.383287 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:56.383288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:56.383406 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:56.383469 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:56.383676 (+ 207us) server_negotiation.cc:299] Negotiation successful
0926 14:58:56.383771 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:58:56.384018 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.376185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:44445)
0926 14:58:56.376327 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:56.376362 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:56.376764 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:56.376967 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:56.376970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:56.377501 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:58:56.377670 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:56.377674 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.378314 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.378314 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:56.379057 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:56.379059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.379381 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.379382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:56.379472 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.379873 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:58:56.379881 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:58:56.381098 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:58:56.382710 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:56.382716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:56.382718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:56.382910 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:56.383121 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:58:56.383122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:58:56.383122 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:58:56.383230 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:58:56.383465 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:58:56.383468 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:58:56.383570 (+ 102us) client_negotiation.cc:769] Sending connection context
0926 14:58:56.383646 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 14:58:56.383812 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260926 14:58:56.384485 23644 heartbeater.cc:347] Connected to a master server at 127.7.37.254:42591
I20260926 14:58:56.384565 23644 heartbeater.cc:464] Registering TS with master...
I20260926 14:58:56.384713 23644 heartbeater.cc:511] Master 127.7.37.254:42591 requested a full tablet report, sending...
I20260926 14:58:56.385063 23181 ts_manager.cc:194] Registered new tserver with Master: c383d5bbca814458b9ab7a2a62ad3321 (127.7.37.195:40525)
I20260926 14:58:56.385388 23181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:44445
I20260926 14:58:56.387897 23648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.382203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35132 (local address 127.7.37.254:42591)
0926 14:58:56.382301 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:56.382303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:56.382390 (+ 87us) server_negotiation.cc:407] Connection header received
0926 14:58:56.382454 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:56.382455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:56.382468 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:56.382524 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:56.383093 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.383645 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.384525 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.384814 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.384882 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.386044 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:56.386045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:56.386045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:56.386051 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:56.387064 (+ 1013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:56.387360 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:56.387360 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:56.387361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:56.387404 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:56.387524 (+ 120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:56.387525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:56.387525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:56.387600 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:56.387716 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:56.387766 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 14:58:56.387837 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":49,"threads_started":1}
I20260926 14:58:56.388471 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 14:58:56.390687 23648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.388752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35144 (local address 127.7.37.254:42591)
0926 14:58:56.388793 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:56.388795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:56.388849 (+ 54us) server_negotiation.cc:407] Connection header received
0926 14:58:56.388901 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:56.388902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:56.388925 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:56.388977 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:58:56.389048 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.389456 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.390098 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.390270 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.390309 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.390402 (+ 93us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:58:56.390434 (+ 32us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0926 14:58:56.390623 (+ 189us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35144: token expired
Metrics: {"server-negotiator.queue_time_us":25}
W20260926 14:58:56.390682 23112 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.37.254:42591)
W20260926 14:58:56.390800 23648 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35144: token expired
I20260926 14:58:56.396778 23648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.390951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35156 (local address 127.7.37.254:42591)
0926 14:58:56.390995 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:56.390996 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:56.391003 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:58:56.391020 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:56.391020 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:56.391028 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:56.391074 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:58:56.391628 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.392110 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.393008 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.393199 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.393249 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.394393 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:58:56.394394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:58:56.394394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:58:56.394401 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:58:56.395818 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:56.396088 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:56.396089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:56.396089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:56.396139 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:58:56.396311 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:58:56.396312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:58:56.396312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:58:56.396423 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:58:56.396506 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:56.396628 (+ 122us) server_negotiation.cc:299] Negotiation successful
0926 14:58:56.396705 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 14:58:56.398087 23181 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:58:56.399812 23181 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:58:56.401598 23442 tablet_service.cc:1467] Processing CreateTablet for tablet 1f3043217fad481c95d7a864491d0354 (DEFAULT_TABLE table=test-table [id=50d997989214406eb3d97fd15636b49b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:58:56.401669 23295 tablet_service.cc:1467] Processing CreateTablet for tablet 1f3043217fad481c95d7a864491d0354 (DEFAULT_TABLE table=test-table [id=50d997989214406eb3d97fd15636b49b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:58:56.401759 23442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f3043217fad481c95d7a864491d0354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:56.401839 23295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f3043217fad481c95d7a864491d0354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:56.402860 23400 tablet_bootstrap.cc:492] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8: Bootstrap starting.
I20260926 14:58:56.403167 23400 tablet_bootstrap.cc:654] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:56.403282 23261 tablet_bootstrap.cc:492] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06: Bootstrap starting.
I20260926 14:58:56.403546 23261 tablet_bootstrap.cc:654] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:56.403617 23400 tablet_bootstrap.cc:492] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8: No bootstrap required, opened a new log
I20260926 14:58:56.403664 23400 ts_tablet_manager.cc:1397] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260926 14:58:56.403822 23400 raft_consensus.cc:348] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } }
I20260926 14:58:56.403898 23400 raft_consensus.cc:374] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:56.403949 23400 raft_consensus.cc:729] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2342907514074c7fb83f34ca3dfafbe8, State: Initialized, Role: FOLLOWER
I20260926 14:58:56.403999 23400 consensus_queue.cc:260] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } }
I20260926 14:58:56.404086 23400 ts_tablet_manager.cc:1428] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 14:58:56.404145 23261 tablet_bootstrap.cc:492] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06: No bootstrap required, opened a new log
I20260926 14:58:56.404204 23261 ts_tablet_manager.cc:1397] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260926 14:58:56.404512 23261 raft_consensus.cc:348] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } }
I20260926 14:58:56.404584 23261 raft_consensus.cc:374] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:56.404629 23261 raft_consensus.cc:729] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46d932a68cc240ea8d6667a10e36bb06, State: Initialized, Role: FOLLOWER
I20260926 14:58:56.404676 23261 consensus_queue.cc:260] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } }
I20260926 14:58:56.404762 23261 ts_tablet_manager.cc:1428] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:58:56.404920 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.401354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:40525 (local address 127.0.0.1:43658)
0926 14:58:56.401383 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:56.401392 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:56.401422 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:56.401731 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:56.401732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:56.401734 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:56.401766 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:56.401768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.402898 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.402898 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:56.403880 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:56.403882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.404709 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.404710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:56.404772 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.404775 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:58:56.404828 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 14:58:56.404865 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14}
I20260926 14:58:56.405012 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.401424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43658 (local address 127.7.37.195:40525)
0926 14:58:56.401559 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:56.401562 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:56.401572 (+ 10us) server_negotiation.cc:407] Connection header received
0926 14:58:56.401602 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:56.401604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:56.401618 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:56.401831 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:56.401896 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.402805 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.403962 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.404630 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.404710 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.404765 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:56.404917 (+ 152us) server_negotiation.cc:299] Negotiation successful
0926 14:58:56.404947 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260926 14:58:56.405578 23578 tablet_service.cc:1467] Processing CreateTablet for tablet 1f3043217fad481c95d7a864491d0354 (DEFAULT_TABLE table=test-table [id=50d997989214406eb3d97fd15636b49b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:58:56.405827 23578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f3043217fad481c95d7a864491d0354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:58:56.408310 23653 tablet_bootstrap.cc:492] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: Bootstrap starting.
I20260926 14:58:56.408838 23653 tablet_bootstrap.cc:654] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: Neither blocks nor log segments found. Creating new log.
I20260926 14:58:56.409296 23653 log.cc:824] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: Log is configured to *not* fsync() on all Append() calls
I20260926 14:58:56.409898 23653 tablet_bootstrap.cc:492] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: No bootstrap required, opened a new log
I20260926 14:58:56.409952 23653 ts_tablet_manager.cc:1397] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:58:56.411281 23653 raft_consensus.cc:348] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } }
I20260926 14:58:56.411408 23653 raft_consensus.cc:374] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:58:56.411438 23653 raft_consensus.cc:729] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c383d5bbca814458b9ab7a2a62ad3321, State: Initialized, Role: FOLLOWER
I20260926 14:58:56.411536 23653 consensus_queue.cc:260] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } }
I20260926 14:58:56.411789 23653 ts_tablet_manager.cc:1428] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:58:56.411841 23644 heartbeater.cc:503] Master 127.7.37.254:42591 was elected leader, sending a full tablet report...
W20260926 14:58:56.484005 23509 tablet.cc:2369] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 14:58:56.625639 23370 tablet.cc:2369] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 14:58:56.626106 23645 tablet.cc:2369] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:58:56.716472 23655 raft_consensus.cc:482] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:58:56.716632 23655 raft_consensus.cc:504] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } }
I20260926 14:58:56.716995 23655 leader_election.cc:288] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46d932a68cc240ea8d6667a10e36bb06 (127.7.37.193:45733), 2342907514074c7fb83f34ca3dfafbe8 (127.7.37.194:39837)
I20260926 14:58:56.719725 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.717292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:49327 (local address 127.7.37.194:39837)
0926 14:58:56.717337 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:56.717339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:56.717347 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:58:56.717377 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:56.717378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:56.717388 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:56.717439 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:56.717589 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.718092 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.719049 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.719402 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.719470 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.719479 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:56.719624 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 14:58:56.719660 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 14:58:56.719727 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.717150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:39837 (local address 127.7.37.195:49327)
0926 14:58:56.717206 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:56.717220 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:56.717285 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:56.717448 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:56.717449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:56.717451 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:56.717524 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:56.717526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.718172 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.718173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:56.718957 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:56.718959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.719477 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.719478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:56.719559 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.719562 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:58:56.719612 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 14:58:56.719652 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":22}
I20260926 14:58:56.719861 23657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.717249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:49197 (local address 127.7.37.193:45733)
0926 14:58:56.717347 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:56.717349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:56.717391 (+ 42us) server_negotiation.cc:407] Connection header received
0926 14:58:56.717464 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:56.717465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:56.717487 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:56.717544 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:58:56.717680 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.718208 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.719141 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.719530 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.719594 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.719666 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:56.719728 (+ 62us) server_negotiation.cc:299] Negotiation successful
0926 14:58:56.719787 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":39,"threads_started":1}
I20260926 14:58:56.720039 23656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.717158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:45733 (local address 127.7.37.195:49197)
0926 14:58:56.717333 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:56.717342 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:58:56.717402 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:58:56.717549 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:58:56.717550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:58:56.717552 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:58:56.717620 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:56.717622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.718282 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.718282 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:56.719057 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:58:56.719060 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.719599 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.719600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:58:56.719665 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.719670 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 14:58:56.719724 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 14:58:56.719967 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":37,"thread_start_us":81,"threads_started":1}
I20260926 14:58:56.720065 23462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3043217fad481c95d7a864491d0354" candidate_uuid: "c383d5bbca814458b9ab7a2a62ad3321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2342907514074c7fb83f34ca3dfafbe8" is_pre_election: true
I20260926 14:58:56.720227 23462 raft_consensus.cc:2454] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c383d5bbca814458b9ab7a2a62ad3321 in term 0.
I20260926 14:58:56.720278 23323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3043217fad481c95d7a864491d0354" candidate_uuid: "c383d5bbca814458b9ab7a2a62ad3321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46d932a68cc240ea8d6667a10e36bb06" is_pre_election: true
I20260926 14:58:56.720396 23323 raft_consensus.cc:2454] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c383d5bbca814458b9ab7a2a62ad3321 in term 0.
I20260926 14:58:56.720435 23533 leader_election.cc:302] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2342907514074c7fb83f34ca3dfafbe8, c383d5bbca814458b9ab7a2a62ad3321; no voters:
I20260926 14:58:56.720556 23655 raft_consensus.cc:2781] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:58:56.720613 23655 raft_consensus.cc:482] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:58:56.720643 23655 raft_consensus.cc:3037] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:56.721309 23655 raft_consensus.cc:504] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } }
I20260926 14:58:56.721421 23655 leader_election.cc:288] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [CANDIDATE]: Term 1 election: Requested vote from peers 46d932a68cc240ea8d6667a10e36bb06 (127.7.37.193:45733), 2342907514074c7fb83f34ca3dfafbe8 (127.7.37.194:39837)
I20260926 14:58:56.721561 23323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3043217fad481c95d7a864491d0354" candidate_uuid: "c383d5bbca814458b9ab7a2a62ad3321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46d932a68cc240ea8d6667a10e36bb06"
I20260926 14:58:56.721626 23323 raft_consensus.cc:3037] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:56.721621 23462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3043217fad481c95d7a864491d0354" candidate_uuid: "c383d5bbca814458b9ab7a2a62ad3321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2342907514074c7fb83f34ca3dfafbe8"
I20260926 14:58:56.721735 23462 raft_consensus.cc:3037] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:58:56.722272 23323 raft_consensus.cc:2454] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c383d5bbca814458b9ab7a2a62ad3321 in term 1.
I20260926 14:58:56.722429 23532 leader_election.cc:302] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46d932a68cc240ea8d6667a10e36bb06, c383d5bbca814458b9ab7a2a62ad3321; no voters:
I20260926 14:58:56.722453 23462 raft_consensus.cc:2454] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c383d5bbca814458b9ab7a2a62ad3321 in term 1.
I20260926 14:58:56.722532 23655 raft_consensus.cc:2781] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:58:56.722664 23655 raft_consensus.cc:686] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [term 1 LEADER]: Becoming Leader. State: Replica: c383d5bbca814458b9ab7a2a62ad3321, State: Running, Role: LEADER
I20260926 14:58:56.722747 23655 consensus_queue.cc:237] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } }
I20260926 14:58:56.723508 23181 catalog_manager.cc:5495] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 reported cstate change: term changed from 0 to 1, leader changed from <none> to c383d5bbca814458b9ab7a2a62ad3321 (127.7.37.195). New cstate: current_term: 1 leader_uuid: "c383d5bbca814458b9ab7a2a62ad3321" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } health_report { overall_health: UNKNOWN } } }
I20260926 14:58:56.754817 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.752611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43662 (local address 127.7.37.195:40525)
0926 14:58:56.752655 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 14:58:56.752657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:58:56.752681 (+ 24us) server_negotiation.cc:407] Connection header received
0926 14:58:56.752766 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:58:56.752767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:58:56.752784 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:58:56.752846 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:58:56.752962 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.753423 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.754141 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:58:56.754320 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:58:56.754380 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:58:56.754516 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:58:56.754611 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:58:56.754686 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 14:58:56.754730 (+ 44us) server_negotiation.cc:299] Negotiation successful
0926 14:58:56.754751 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:58:56.756353 23462 raft_consensus.cc:1261] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8 [term 1 FOLLOWER]: Refusing update from remote peer c383d5bbca814458b9ab7a2a62ad3321: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:58:56.756363 23323 raft_consensus.cc:1261] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [term 1 FOLLOWER]: Refusing update from remote peer c383d5bbca814458b9ab7a2a62ad3321: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:58:56.756687 23658 consensus_queue.cc:1035] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:58:56.756806 23655 consensus_queue.cc:1035] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:58:56.758556 23660 mvcc.cc:205] Tried to move back new op lower bound from 7333620681751891968 to 7333620681616887808. Current Snapshot: MvccSnapshot[applied={T|T < 7333620681751891968}]
I20260926 14:58:56.759388 23662 mvcc.cc:205] Tried to move back new op lower bound from 7333620681751891968 to 7333620681616887808. Current Snapshot: MvccSnapshot[applied={T|T < 7333620681751891968 or (T in {7333620681751891968})}]
I20260926 14:58:56.760660 23180 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:58:56.760746 23180 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:58:56.761402 23180 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6: Sending DeleteTablet for 3 replicas of tablet 1f3043217fad481c95d7a864491d0354
I20260926 14:58:56.761664 23295 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f3043217fad481c95d7a864491d0354 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:58:56 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:33716
I20260926 14:58:56.761725 23578 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f3043217fad481c95d7a864491d0354 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:58:56 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:43658
I20260926 14:58:56.761844 23442 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f3043217fad481c95d7a864491d0354 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:58:56 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:39282
I20260926 14:58:56.761868 23670 tablet_replica.cc:330] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06: stopping tablet replica
I20260926 14:58:56.761934 23670 raft_consensus.cc:2229] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:58:56.761955 23671 tablet_replica.cc:330] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: stopping tablet replica
I20260926 14:58:56.761986 23670 raft_consensus.cc:2258] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:58:56.762075 23671 raft_consensus.cc:2229] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:58:56.762087 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 23220
I20260926 14:58:56.762190 23671 raft_consensus.cc:2258] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:58:56.762584 23671 ts_tablet_manager.cc:1905] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:58:56.764328 23671 ts_tablet_manager.cc:1918] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:58:56.764422 23671 log.cc:1196] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/1f3043217fad481c95d7a864491d0354
I20260926 14:58:56.764685 23671 ts_tablet_manager.cc:1939] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: Deleting consensus metadata
I20260926 14:58:56.765049 23167 catalog_manager.cc:4841] TS c383d5bbca814458b9ab7a2a62ad3321 (127.7.37.195:40525): tablet 1f3043217fad481c95d7a864491d0354 (table test-table [id=50d997989214406eb3d97fd15636b49b]) successfully deleted
W20260926 14:58:56.766897 23166 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.37.193:45733 (error 108)
I20260926 14:58:56.767204 23672 tablet_replica.cc:330] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8: stopping tablet replica
I20260926 14:58:56.767256 23672 raft_consensus.cc:2229] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:58:56.767297 23672 raft_consensus.cc:2258] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:58:56.767346 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.767225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:45733 (local address 127.0.0.1:33724)
0926 14:58:56.767274 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:56.767316 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.193:45733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260926 14:58:56.767441 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 23373
W20260926 14:58:56.767467 23166 catalog_manager.cc:4584] TS 46d932a68cc240ea8d6667a10e36bb06 (127.7.37.193:45733): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1f3043217fad481c95d7a864491d0354: Network error: Client connection negotiation failed: client connection to 127.7.37.193:45733: connect: Connection refused (error 111)
I20260926 14:58:56.771471 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:58:56.771381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:39837 (local address 127.0.0.1:39284)
0926 14:58:56.771425 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 14:58:56.771450 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.194:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260926 14:58:56.772215 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 23512
I20260926 14:58:56.776160 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 23148
I20260926 14:59:16.780256 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:42591
--webserver_interface=127.7.37.254
--webserver_port=39927
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:42591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:16.856074 23674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:16.856259 23674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:16.856290 23674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:16.857398 23674 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 14:59:16.857438 23674 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 14:59:16.857460 23674 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 14:59:16.857477 23674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:16.857497 23674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:16.857522 23674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:16.857548 23674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:16.857576 23674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:16.859020 23674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:42591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:42591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=39927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:16.859246 23674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:16.859439 23674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:16.863925 23680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:16.864068 23674 server_base.cc:1034] running on GCE node
W20260926 14:59:16.863905 23682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:16.863902 23679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:16.864385 23674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:16.864629 23674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:16.865760 23674 hybrid_clock.cc:648] HybridClock initialized: now 1790434756865744 us; error 35 us; skew 500 ppm
Sep 26 14:59:16 dist-test-slave-f9xb krb5kdc[22604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434756, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:16.867997 23674 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:59:16.868360 23674 webserver.cc:460] Webserver started at http://127.7.37.254:39927/ using document root <none> and password file <none>
I20260926 14:59:16.868534 23674 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:16.868579 23674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:16.869760 23674 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 14:59:16.870339 23689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:16.870491 23674 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:59:16.870561 23674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "42baa97c88214e84bad7f2f0421205a6"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "a531b386fb169ea4c3bb41fb5ee5ec4e"
server_key_iv: "b60a46bb38b6fa2fdbacf66a96da5a6a"
server_key_version: "encryptionkey@0"
I20260926 14:59:16.870828 23674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:16.885252 23674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:16.887065 23674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:16.887185 23674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:16.890528 23674 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:42591
I20260926 14:59:16.890559 23741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:42591 every 8 connection(s)
I20260926 14:59:16.890849 23674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:16.892100 23742 sys_catalog.cc:263] Verifying existing consensus state
I20260926 14:59:16.892657 23742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6: Bootstrap starting.
I20260926 14:59:16.894408 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 23674
I20260926 14:59:16.894639 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:45733
--local_ip_for_outbound_sockets=127.7.37.193
--tserver_master_addrs=127.7.37.254:42591
--webserver_port=39373
--webserver_interface=127.7.37.193
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 14:59:16.900575 23742 log.cc:824] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:16.902482 23742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 14:59:16.902719 23742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6: Bootstrap complete.
I20260926 14:59:16.904594 23742 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42baa97c88214e84bad7f2f0421205a6" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42591 } }
I20260926 14:59:16.904790 23742 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42baa97c88214e84bad7f2f0421205a6, State: Initialized, Role: FOLLOWER
I20260926 14:59:16.904923 23742 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42baa97c88214e84bad7f2f0421205a6" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42591 } }
I20260926 14:59:16.904978 23742 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:16.905015 23742 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:16.905058 23742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 2 FOLLOWER]: Advancing to term 3
I20260926 14:59:16.905908 23742 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42baa97c88214e84bad7f2f0421205a6" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42591 } }
I20260926 14:59:16.905988 23742 leader_election.cc:302] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42baa97c88214e84bad7f2f0421205a6; no voters:
I20260926 14:59:16.906131 23742 leader_election.cc:288] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 14:59:16.906183 23746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 14:59:16.906293 23746 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [term 3 LEADER]: Becoming Leader. State: Replica: 42baa97c88214e84bad7f2f0421205a6, State: Running, Role: LEADER
I20260926 14:59:16.906358 23742 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:16.906456 23746 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42baa97c88214e84bad7f2f0421205a6" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42591 } }
I20260926 14:59:16.906708 23748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "42baa97c88214e84bad7f2f0421205a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42baa97c88214e84bad7f2f0421205a6" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42591 } } }
I20260926 14:59:16.906961 23748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:16.906903 23747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42baa97c88214e84bad7f2f0421205a6. Latest consensus state: current_term: 3 leader_uuid: "42baa97c88214e84bad7f2f0421205a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42baa97c88214e84bad7f2f0421205a6" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 42591 } } }
I20260926 14:59:16.907192 23747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:16.907286 23756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:16.908011 23756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:16.908155 23756 catalog_manager.cc:1259] Loaded cluster ID: 1cb4114710e146e0aa7d4e0c2d3cefcc
I20260926 14:59:16.908294 23756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:59:16.909365 23756 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:16.909509 23756 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6: Loaded TSK: 0
I20260926 14:59:16.909541 23756 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 42baa97c88214e84bad7f2f0421205a6: Loaded TSK: 1
F20260926 14:59:16.909755 23756 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fed3c5c29ad google::LogMessage::Fail() at ??:0
@ 0x7fed3c5c588a google::LogMessage::SendToLog() at ??:0
@ 0x7fed3c5c24de google::LogMessage::Flush() at ??:0
@ 0x7fed3c5c3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fed3ed2f104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fed3ed2cd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fed4451c1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fed4451c48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fed4451c594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fed444e2c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fed444ecad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fed3e4900d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fed3e485954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fed3f2e96db start_thread at ??:0
@ 0x7fed3b0c061f clone at ??:0
W20260926 14:59:16.985932 23744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:16.986092 23744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:16.986112 23744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:16.987215 23744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:16.987251 23744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:16.987275 23744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:59:16.988636 23744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:45733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=39373
--tserver_master_addrs=127.7.37.254:42591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:16.988839 23744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:16.989020 23744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:16.993340 23786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:16.993356 23784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:16.993849 23783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:16.994087 23744 server_base.cc:1034] running on GCE node
I20260926 14:59:16.994261 23744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:16.994455 23744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:16.995622 23744 hybrid_clock.cc:648] HybridClock initialized: now 1790434756995607 us; error 29 us; skew 500 ppm
Sep 26 14:59:16 dist-test-slave-f9xb krb5kdc[22604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434756, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:16.997836 23744 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:59:16.998238 23744 webserver.cc:460] Webserver started at http://127.7.37.193:39373/ using document root <none> and password file <none>
I20260926 14:59:16.998438 23744 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:16.998479 23744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:16.999629 23744 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 14:59:17.000219 23793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:17.000365 23744 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:17.000458 23744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "46d932a68cc240ea8d6667a10e36bb06"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "45349e80f48bda9539148412f18bc074"
server_key_iv: "09696fb6e6a916a91cdbe2e27c777dbe"
server_key_version: "encryptionkey@0"
I20260926 14:59:17.000833 23744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:17.020056 23744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:17.022189 23744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:17.022331 23744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:17.022547 23744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:17.022985 23800 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 14:59:17.024308 23744 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 14:59:17.024363 23744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260926 14:59:17.024392 23744 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 14:59:17.025157 23744 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 14:59:17.025200 23744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 14:59:17.025269 23800 tablet_bootstrap.cc:492] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06: Bootstrap starting.
I20260926 14:59:17.031769 23744 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:45733
I20260926 14:59:17.031911 23907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:45733 every 8 connection(s)
I20260926 14:59:17.032192 23744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:59:17.034807 23800 log.cc:824] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:17.036729 23800 tablet_bootstrap.cc:492] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 14:59:17.036949 23800 tablet_bootstrap.cc:492] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06: Bootstrap complete.
I20260926 14:59:17.037640 23800 ts_tablet_manager.cc:1397] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260926 14:59:17.038556 23800 raft_consensus.cc:348] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } }
I20260926 14:59:17.038763 23800 raft_consensus.cc:729] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46d932a68cc240ea8d6667a10e36bb06, State: Initialized, Role: FOLLOWER
I20260926 14:59:17.038877 23800 consensus_queue.cc:260] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } }
I20260926 14:59:17.039083 23800 ts_tablet_manager.cc:1428] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20260926 14:59:17.039269 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 23744
I20260926 14:59:17.039533 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:39837
--local_ip_for_outbound_sockets=127.7.37.194
--tserver_master_addrs=127.7.37.254:42591
--webserver_port=45659
--webserver_interface=127.7.37.194
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:17.097422 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 14:59:17.032842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:54047)
0926 14:59:17.033025 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:17.033071 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:17.033666 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:17.097266 (+ 63600us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":94,"threads_started":1}
W20260926 14:59:17.097658 23908 heartbeater.cc:644] Failed to heartbeat to 127.7.37.254:42591 (0 consecutive failures): Network error: Failed to ping master at 127.7.37.254:42591: Client connection negotiation failed: client connection to 127.7.37.254:42591: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 14:59:17.097970 23910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:17.097874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:54407)
0926 14:59:17.097916 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:17.097950 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260926 14:59:17.098280 23910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:17.098204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:38613)
0926 14:59:17.098245 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:17.098267 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 14:59:17.098376 23908 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 14:59:17.124403 23913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:17.124575 23913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:17.124596 23913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:17.125715 23913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:17.125748 23913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:17.125772 23913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:59:17.127139 23913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:39837
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=45659
--tserver_master_addrs=127.7.37.254:42591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:17.127314 23913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:17.127470 23913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 14:59:17.131909 23913 server_base.cc:1034] running on GCE node
W20260926 14:59:17.131981 23921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:17.131899 23919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:17.131886 23918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:17.132275 23913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:17.132485 23913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:17.133617 23913 hybrid_clock.cc:648] HybridClock initialized: now 1790434757133604 us; error 26 us; skew 500 ppm
Sep 26 14:59:17 dist-test-slave-f9xb krb5kdc[22604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434757, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:17.135710 23913 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:59:17.136099 23913 webserver.cc:460] Webserver started at http://127.7.37.194:45659/ using document root <none> and password file <none>
I20260926 14:59:17.136283 23913 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:17.136322 23913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:17.137427 23913 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:59:17.137917 23928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:17.138048 23913 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:17.138121 23913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "2342907514074c7fb83f34ca3dfafbe8"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "01f8074bb4b5e3f8764c3be08f04e081"
server_key_iv: "2b3f55e1bbdab8e0ae5519ec27cbdec8"
server_key_version: "encryptionkey@0"
I20260926 14:59:17.138405 23913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:17.148782 23913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:17.150321 23913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:17.150432 23913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:17.150599 23913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:17.150942 23935 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 14:59:17.152267 23913 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 14:59:17.152312 23913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260926 14:59:17.152341 23913 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 14:59:17.152874 23913 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 14:59:17.152904 23913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 14:59:17.152984 23935 tablet_bootstrap.cc:492] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8: Bootstrap starting.
I20260926 14:59:17.158998 23913 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:39837
I20260926 14:59:17.159053 24042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:39837 every 8 connection(s)
I20260926 14:59:17.159381 23913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:59:17.160322 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:17.159970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:55559)
0926 14:59:17.160144 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:17.160202 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
W20260926 14:59:17.160517 24043 heartbeater.cc:644] Failed to heartbeat to 127.7.37.254:42591 (0 consecutive failures): Network error: Failed to ping master at 127.7.37.254:42591: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
I20260926 14:59:17.160779 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:17.160700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:48881)
0926 14:59:17.160740 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:17.160763 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260926 14:59:17.161041 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:17.160966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:36757)
0926 14:59:17.161001 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:17.161022 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260926 14:59:17.161151 24043 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 14:59:17.161312 23935 log.cc:824] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:17.162971 23935 tablet_bootstrap.cc:492] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 14:59:17.163162 23935 tablet_bootstrap.cc:492] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8: Bootstrap complete.
I20260926 14:59:17.163887 23935 ts_tablet_manager.cc:1397] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.009s
I20260926 14:59:17.164927 23935 raft_consensus.cc:348] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } }
I20260926 14:59:17.165143 23935 raft_consensus.cc:729] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2342907514074c7fb83f34ca3dfafbe8, State: Initialized, Role: FOLLOWER
I20260926 14:59:17.165231 23935 consensus_queue.cc:260] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } }
I20260926 14:59:17.165392 23935 ts_tablet_manager.cc:1428] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260926 14:59:17.165599 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:17.165524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:47649)
0926 14:59:17.165559 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:17.165583 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260926 14:59:17.165799 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:17.165741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:58743)
0926 14:59:17.165769 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:17.165787 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 14:59:17.167218 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 23913
I20260926 14:59:17.167429 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:40525
--local_ip_for_outbound_sockets=127.7.37.195
--tserver_master_addrs=127.7.37.254:42591
--webserver_port=45983
--webserver_interface=127.7.37.195
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:17.242846 24048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:17.243011 24048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:17.243031 24048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:17.244092 24048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:17.244125 24048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:17.244148 24048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:59:17.245517 24048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:40525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=45983
--tserver_master_addrs=127.7.37.254:42591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:17.245675 24048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:17.245829 24048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 14:59:17.250284 24048 server_base.cc:1034] running on GCE node
W20260926 14:59:17.250244 24056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:17.250267 24054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:17.250253 24053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:17.250607 24048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:17.250834 24048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:17.251958 24048 hybrid_clock.cc:648] HybridClock initialized: now 1790434757251949 us; error 26 us; skew 500 ppm
Sep 26 14:59:17 dist-test-slave-f9xb krb5kdc[22604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434757, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:17.254055 24048 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:59:17.254437 24048 webserver.cc:460] Webserver started at http://127.7.37.195:45983/ using document root <none> and password file <none>
I20260926 14:59:17.254626 24048 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:17.254670 24048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:17.255862 24048 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:59:17.256352 24063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:17.256488 24048 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:17.256560 24048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "c383d5bbca814458b9ab7a2a62ad3321"
format_stamp: "Formatted at 2026-09-26 14:58:35 on dist-test-slave-f9xb"
server_key: "445e874ad342de95903f1ecccc7e9658"
server_key_iv: "59209239c92b50d19579d14d15cbe76b"
server_key_version: "encryptionkey@0"
I20260926 14:59:17.256827 24048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:17.267920 24048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:17.269565 24048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:17.269685 24048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:17.269876 24048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:17.270112 24048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:59:17.270140 24048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:17.270166 24048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:59:17.270188 24048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:17.275301 24048 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:40525
I20260926 14:59:17.275353 24176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:40525 every 8 connection(s)
I20260926 14:59:17.275615 24048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:59:17.276451 24179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:17.276150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:47105)
0926 14:59:17.276277 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:17.276333 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":67,"threads_started":1}
W20260926 14:59:17.276758 24177 heartbeater.cc:644] Failed to heartbeat to 127.7.37.254:42591 (0 consecutive failures): Network error: Failed to ping master at 127.7.37.254:42591: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
I20260926 14:59:17.277006 24179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:17.276933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:58405)
0926 14:59:17.276968 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:17.276991 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260926 14:59:17.277218 24179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:17.277155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:48947)
0926 14:59:17.277189 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:17.277207 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 14:59:17.277292 24177 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 14:59:17.281492 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 24048
I20260926 14:59:18.098914 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:18.098691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:59803)
0926 14:59:18.098821 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:18.098856 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":56,"threads_started":1}
I20260926 14:59:18.166275 24182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:18.166097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:57353)
0926 14:59:18.166209 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:18.166240 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
I20260926 14:59:18.189606 24183 raft_consensus.cc:482] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:59:18.189720 24183 raft_consensus.cc:504] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } }
I20260926 14:59:18.190025 24183 leader_election.cc:288] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c383d5bbca814458b9ab7a2a62ad3321 (127.7.37.195:40525), 2342907514074c7fb83f34ca3dfafbe8 (127.7.37.194:39837)
Sep 26 14:59:18 dist-test-slave-f9xb krb5kdc[22604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434756, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.194@KRBTEST.COM
Sep 26 14:59:18 dist-test-slave-f9xb krb5kdc[22604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434756, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.195@KRBTEST.COM
I20260926 14:59:18.202020 24185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:18.190283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:39837 (local address 127.7.37.193:46793)
0926 14:59:18.190383 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:18.190392 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:18.190428 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:18.190849 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:18.190852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:18.191220 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:18.191400 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:18.191406 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:18.192460 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:18.192461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:18.193271 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:18.193274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:18.193793 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:18.193794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:18.193899 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:18.197122 (+ 3223us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:59:18.197130 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:18.198557 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:18.200615 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:18.200621 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:18.200623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:18.200862 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:18.201115 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:18.201116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:18.201116 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:18.201205 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:18.201552 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:18.201558 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:18.201668 (+ 110us) client_negotiation.cc:769] Sending connection context
0926 14:59:18.201751 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 14:59:18.201926 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":43,"thread_start_us":51,"threads_started":1}
I20260926 14:59:18.202037 24186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:18.190362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:46793 (local address 127.7.37.194:39837)
0926 14:59:18.190495 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:18.190505 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:18.190526 (+ 21us) server_negotiation.cc:407] Connection header received
0926 14:59:18.190629 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:18.190632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:18.190683 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:18.190793 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:18.191493 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:18.192377 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:18.193346 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:18.193722 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:18.193813 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:18.198657 (+ 4844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:18.198661 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:18.198661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:18.198671 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:18.200509 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:18.200952 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:18.200956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:18.200956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:18.201059 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:18.201277 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:18.201278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:18.201278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:18.201491 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:18.201558 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:18.201806 (+ 248us) server_negotiation.cc:299] Negotiation successful
0926 14:59:18.201918 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":51,"threads_started":1}
I20260926 14:59:18.202049 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:18.190259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:46603 (local address 127.7.37.195:40525)
0926 14:59:18.190416 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:18.190427 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:18.190453 (+ 26us) server_negotiation.cc:407] Connection header received
0926 14:59:18.190913 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:18.190915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:18.190965 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:18.191095 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:18.191503 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:18.192393 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:18.193347 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:18.193723 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:18.193820 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:18.199013 (+ 5193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:18.199016 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:18.199017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:18.199026 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:18.200751 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:18.201074 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:18.201076 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:18.201076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:18.201160 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:18.201351 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:18.201352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:18.201352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:18.201576 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:18.201650 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:18.201822 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 14:59:18.201950 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":61,"threads_started":1}
I20260926 14:59:18.202122 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:18.190168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:40525 (local address 127.7.37.193:46603)
0926 14:59:18.190221 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:18.190232 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:18.190292 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:18.191087 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:18.191088 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:18.191284 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:18.191400 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:18.191406 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:18.192471 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:18.192472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:18.193270 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:18.193272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:18.193796 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:18.193797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:18.193899 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:18.197135 (+ 3236us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:59:18.197140 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:18.198927 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:18.200827 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:18.200827 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:18.200828 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:18.201001 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:18.201228 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:18.201229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:18.201229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:18.201295 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:18.201641 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:18.201642 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:18.201686 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 14:59:18.201758 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 14:59:18.202000 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":171}
I20260926 14:59:18.202780 23997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3043217fad481c95d7a864491d0354" candidate_uuid: "46d932a68cc240ea8d6667a10e36bb06" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2342907514074c7fb83f34ca3dfafbe8" is_pre_election: true
I20260926 14:59:18.202920 23997 raft_consensus.cc:2454] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46d932a68cc240ea8d6667a10e36bb06 in term 1.
W20260926 14:59:18.203145 23796 leader_election.cc:341] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c383d5bbca814458b9ab7a2a62ad3321 (127.7.37.195:40525): Not found: Tablet not found: 1f3043217fad481c95d7a864491d0354
I20260926 14:59:18.203235 23796 leader_election.cc:302] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2342907514074c7fb83f34ca3dfafbe8, 46d932a68cc240ea8d6667a10e36bb06; no voters: c383d5bbca814458b9ab7a2a62ad3321
I20260926 14:59:18.203344 24183 raft_consensus.cc:2781] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 14:59:18.203401 24183 raft_consensus.cc:482] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:59:18.203416 24183 raft_consensus.cc:3037] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [term 1 FOLLOWER]: Advancing to term 2
I20260926 14:59:18.204483 24183 raft_consensus.cc:504] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } }
I20260926 14:59:18.204609 24183 leader_election.cc:288] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [CANDIDATE]: Term 2 election: Requested vote from peers c383d5bbca814458b9ab7a2a62ad3321 (127.7.37.195:40525), 2342907514074c7fb83f34ca3dfafbe8 (127.7.37.194:39837)
I20260926 14:59:18.204800 23997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3043217fad481c95d7a864491d0354" candidate_uuid: "46d932a68cc240ea8d6667a10e36bb06" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2342907514074c7fb83f34ca3dfafbe8"
W20260926 14:59:18.204877 23796 leader_election.cc:341] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c383d5bbca814458b9ab7a2a62ad3321 (127.7.37.195:40525): Not found: Tablet not found: 1f3043217fad481c95d7a864491d0354
I20260926 14:59:18.204887 23997 raft_consensus.cc:3037] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8 [term 1 FOLLOWER]: Advancing to term 2
I20260926 14:59:18.205948 23997 raft_consensus.cc:2454] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46d932a68cc240ea8d6667a10e36bb06 in term 2.
I20260926 14:59:18.206096 23796 leader_election.cc:302] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2342907514074c7fb83f34ca3dfafbe8, 46d932a68cc240ea8d6667a10e36bb06; no voters: c383d5bbca814458b9ab7a2a62ad3321
I20260926 14:59:18.206179 24183 raft_consensus.cc:2781] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 14:59:18.206311 24183 raft_consensus.cc:686] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [term 2 LEADER]: Becoming Leader. State: Replica: 46d932a68cc240ea8d6667a10e36bb06, State: Running, Role: LEADER
I20260926 14:59:18.206395 24183 consensus_queue.cc:237] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } }
I20260926 14:59:18.206555 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:18.206464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:44299)
0926 14:59:18.206504 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:18.206532 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260926 14:59:18.277665 24189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:18.277524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:44515)
0926 14:59:18.277602 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:18.277636 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66,"thread_start_us":42,"threads_started":1}
I20260926 14:59:18.589946 23997 raft_consensus.cc:1261] T 1f3043217fad481c95d7a864491d0354 P 2342907514074c7fb83f34ca3dfafbe8 [term 2 FOLLOWER]: Refusing update from remote peer 46d932a68cc240ea8d6667a10e36bb06: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260926 14:59:18.590261 24183 consensus_queue.cc:1035] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260926 14:59:18.590984 24182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:18.590887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:36745)
0926 14:59:18.590932 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:18.590955 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260926 14:59:18.591770 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:18.591658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:59449)
0926 14:59:18.591696 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:18.591720 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260926 14:59:18.591931 23796 consensus_peers.cc:487] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06 -> Peer c383d5bbca814458b9ab7a2a62ad3321 (127.7.37.195:40525): Couldn't send request to peer c383d5bbca814458b9ab7a2a62ad3321. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f3043217fad481c95d7a864491d0354. This is attempt 1: this message will repeat every 5th retry.
I20260926 14:59:18.591980 24182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:18.591895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:59203)
0926 14:59:18.591930 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:18.591957 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 14:59:19.101920 24196 ts_tablet_manager.cc:927] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: Initiating tablet copy from peer 46d932a68cc240ea8d6667a10e36bb06 (127.7.37.193:45733)
I20260926 14:59:19.102108 24196 tablet_copy_client.cc:316] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: tablet copy: Beginning tablet copy session from remote peer at address 127.7.37.193:45733
Sep 26 14:59:19 dist-test-slave-f9xb krb5kdc[22604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434757, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.193@KRBTEST.COM
I20260926 14:59:19.109678 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:19.102609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:42797 (local address 127.7.37.193:45733)
0926 14:59:19.102721 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:19.102727 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:19.102741 (+ 14us) server_negotiation.cc:407] Connection header received
0926 14:59:19.102765 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:19.102766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:19.102779 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:19.102843 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:19.103250 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:19.103949 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:19.104719 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:19.105049 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:19.105103 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:19.106992 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:19.106999 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:19.107000 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:19.107006 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:19.108442 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:19.108827 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:19.108830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:19.108831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:19.108892 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:19.109077 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:19.109078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:19.109078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:19.109248 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:19.109318 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:19.109519 (+ 201us) server_negotiation.cc:299] Negotiation successful
0926 14:59:19.109591 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":52,"threads_started":1}
I20260926 14:59:19.109668 24198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:19.102522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:45733 (local address 127.7.37.195:42797)
0926 14:59:19.102652 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:19.102666 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:19.102722 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:19.102850 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:19.102852 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:19.103131 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:19.103186 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:19.103188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:19.104026 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:19.104027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:19.104659 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:19.104662 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:19.105106 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:19.105107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:19.105150 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:19.105598 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:59:19.105609 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:19.106912 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:19.108514 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:19.108520 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:19.108522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:19.108724 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:19.108951 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:19.108951 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:19.108952 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:19.109004 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:19.109332 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:19.109337 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:19.109384 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 14:59:19.109456 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 14:59:19.109564 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":75,"threads_started":1}
I20260926 14:59:19.110090 23882 tablet_copy_service.cc:140] P 46d932a68cc240ea8d6667a10e36bb06: Received BeginTabletCopySession request for tablet 1f3043217fad481c95d7a864491d0354 from peer c383d5bbca814458b9ab7a2a62ad3321 ({username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:42797)
I20260926 14:59:19.110152 23882 tablet_copy_service.cc:161] P 46d932a68cc240ea8d6667a10e36bb06: Beginning new tablet copy session on tablet 1f3043217fad481c95d7a864491d0354 from peer c383d5bbca814458b9ab7a2a62ad3321 at {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:42797: session id = c383d5bbca814458b9ab7a2a62ad3321-1f3043217fad481c95d7a864491d0354
I20260926 14:59:19.110714 23882 tablet_copy_source_session.cc:205] T 1f3043217fad481c95d7a864491d0354 P 46d932a68cc240ea8d6667a10e36bb06: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 14:59:19.111609 24196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f3043217fad481c95d7a864491d0354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:19.114053 24196 tablet_copy_client.cc:791] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: tablet copy: Starting download of 0 data blocks...
I20260926 14:59:19.114176 24196 tablet_copy_client.cc:655] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: tablet copy: Starting download of 1 WAL segments...
I20260926 14:59:19.114877 24196 tablet_copy_client.cc:531] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 14:59:19.115473 24196 tablet_bootstrap.cc:492] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: Bootstrap starting.
I20260926 14:59:19.116719 24196 log.cc:824] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:19.118845 24196 tablet_bootstrap.cc:492] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 14:59:19.119022 24196 tablet_bootstrap.cc:492] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: Bootstrap complete.
I20260926 14:59:19.119127 24196 ts_tablet_manager.cc:1397] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260926 14:59:19.120147 24196 raft_consensus.cc:348] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } }
I20260926 14:59:19.120280 24196 raft_consensus.cc:729] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c383d5bbca814458b9ab7a2a62ad3321, State: Initialized, Role: FOLLOWER
I20260926 14:59:19.120390 24196 consensus_queue.cc:260] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c383d5bbca814458b9ab7a2a62ad3321" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 40525 } } peers { permanent_uuid: "46d932a68cc240ea8d6667a10e36bb06" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45733 } } peers { permanent_uuid: "2342907514074c7fb83f34ca3dfafbe8" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 39837 } }
I20260926 14:59:19.120613 24196 ts_tablet_manager.cc:1428] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321: Time spent starting tablet: real 0.001s user 0.002s sys 0.001s
I20260926 14:59:19.120851 24198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:19.120760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:38369)
0926 14:59:19.120797 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:19.120823 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260926 14:59:19.120920 23882 tablet_copy_service.cc:342] P 46d932a68cc240ea8d6667a10e36bb06: Request end of tablet copy session c383d5bbca814458b9ab7a2a62ad3321-1f3043217fad481c95d7a864491d0354 received from {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:42797
I20260926 14:59:19.120970 23882 tablet_copy_service.cc:433] P 46d932a68cc240ea8d6667a10e36bb06: ending tablet copy session c383d5bbca814458b9ab7a2a62ad3321-1f3043217fad481c95d7a864491d0354 on tablet 1f3043217fad481c95d7a864491d0354 with peer c383d5bbca814458b9ab7a2a62ad3321
I20260926 14:59:19.121117 24198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:19.121041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:38119)
0926 14:59:19.121071 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:19.121098 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260926 14:59:19.485419 24131 raft_consensus.cc:1203] T 1f3043217fad481c95d7a864491d0354 P c383d5bbca814458b9ab7a2a62ad3321 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260926 14:59:19.485512 24203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:19.485327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:44527)
0926 14:59:19.485436 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:19.485469 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":60,"threads_started":1}
I20260926 14:59:19.485859 24198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:19.485755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:52563)
0926 14:59:19.485806 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:19.485831 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 14:59:19.592487 24204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:19.592312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:52217)
0926 14:59:19.592404 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:19.592434 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":47,"threads_started":1}
I20260926 14:59:20.486208 24205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:20.485976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:48391)
0926 14:59:20.486114 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:20.486152 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260926 14:59:20.486431 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:20.486236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:39243)
0926 14:59:20.486348 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:20.486376 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":49,"threads_started":1}
I20260926 14:59:20.593044 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:20.592847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:47369)
0926 14:59:20.592969 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:20.593001 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260926 14:59:21.486805 24208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:21.486626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:46711)
0926 14:59:21.486727 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:21.486759 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":59,"threads_started":1}
I20260926 14:59:21.486930 24209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:21.486766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:47927)
0926 14:59:21.486853 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:21.486884 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72,"thread_start_us":40,"threads_started":1}
I20260926 14:59:21.593637 24210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:21.593458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:48021)
0926 14:59:21.593562 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:21.593595 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":51,"threads_started":1}
I20260926 14:59:22.487411 24211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:22.487210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:36001)
0926 14:59:22.487334 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:22.487364 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260926 14:59:22.487480 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:22.487319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:38717)
0926 14:59:22.487408 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:22.487435 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75,"thread_start_us":42,"threads_started":1}
I20260926 14:59:22.487761 24211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:22.487658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:45695)
0926 14:59:22.487696 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:22.487718 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260926 14:59:22.594265 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:22.594046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:33979)
0926 14:59:22.594190 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:22.594223 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":55,"threads_started":1}
I20260926 14:59:23.488054 24214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:23.487843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:34053)
0926 14:59:23.487964 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:23.488003 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
I20260926 14:59:23.488303 24215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:23.488118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:49737)
0926 14:59:23.488239 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:23.488268 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":51,"threads_started":1}
I20260926 14:59:23.594785 24216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:23.594609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:42487)
0926 14:59:23.594714 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:23.594744 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":63,"threads_started":1}
I20260926 14:59:24.488622 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:24.488406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:40015)
0926 14:59:24.488529 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:24.488568 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
I20260926 14:59:24.488875 24218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:24.488638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:44133)
0926 14:59:24.488805 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:24.488838 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":69,"threads_started":1}
I20260926 14:59:24.595320 24219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:24.595106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:45437)
0926 14:59:24.595238 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:24.595275 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260926 14:59:25.489291 24220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:25.489108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:42011)
0926 14:59:25.489219 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:25.489247 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":61,"threads_started":1}
I20260926 14:59:25.489487 24221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:25.489274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:46337)
0926 14:59:25.489415 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:25.489447 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":69,"threads_started":1}
I20260926 14:59:25.595877 24222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:25.595677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:57783)
0926 14:59:25.595804 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:25.595832 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260926 14:59:26.489895 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:26.489644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:36177)
0926 14:59:26.489786 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:26.489838 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260926 14:59:26.490044 24224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:26.489861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:34161)
0926 14:59:26.489960 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:26.489998 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":56,"threads_started":1}
I20260926 14:59:26.596447 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:26.596228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:48213)
0926 14:59:26.596370 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:26.596402 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260926 14:59:27.490525 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:27.490310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:50533)
0926 14:59:27.490426 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:27.490467 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":54,"threads_started":1}
I20260926 14:59:27.490614 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:27.490428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:44473)
0926 14:59:27.490535 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:27.490567 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":62,"threads_started":1}
I20260926 14:59:27.597023 24228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:27.596846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:42819)
0926 14:59:27.596946 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:27.596977 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":55,"threads_started":1}
I20260926 14:59:28.491151 24229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:28.490952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:59857)
0926 14:59:28.491072 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:28.491103 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260926 14:59:28.491242 24230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:28.491016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:54769)
0926 14:59:28.491160 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:28.491191 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
I20260926 14:59:28.597604 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:28.597402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:38295)
0926 14:59:28.597528 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:28.597559 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260926 14:59:29.491787 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:29.491578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:54153)
0926 14:59:29.491690 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:29.491717 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":55,"threads_started":1}
I20260926 14:59:29.491778 24232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:29.491526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:35031)
0926 14:59:29.491673 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:29.491704 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260926 14:59:29.598136 24234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:29.597968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:47283)
0926 14:59:29.598063 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:29.598093 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":52,"threads_started":1}
I20260926 14:59:30.492319 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:30.492109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:53119)
0926 14:59:30.492228 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:30.492264 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":51,"threads_started":1}
I20260926 14:59:30.492409 24236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:30.492210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:33875)
0926 14:59:30.492332 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:30.492363 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
I20260926 14:59:30.598728 24237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:30.598551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:34831)
0926 14:59:30.598649 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:30.598682 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":54,"threads_started":1}
I20260926 14:59:31.492892 24238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:31.492713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.193:56483)
0926 14:59:31.492817 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:31.492844 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":56,"threads_started":1}
I20260926 14:59:31.492974 24239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:31.492787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.195:52551)
0926 14:59:31.492902 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:31.492929 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
I20260926 14:59:31.599321 24240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:31.599133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:42591 (local address 127.7.37.194:50807)
0926 14:59:31.599242 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:31.599274 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.254:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":59,"threads_started":1}
I20260926 14:59:32.286312 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 23744
I20260926 14:59:32.290406 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 23913
I20260926 14:59:32.294013 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 24048
2026-09-26T14:59:32Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58364 ms)
[----------] 30 tests from SecurityITest (217286 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:32 dist-test-slave-f9xb krb5kdc[24243](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:32 dist-test-slave-f9xb krb5kdc[24243](info): set up 2 sockets
Sep 26 14:59:32 dist-test-slave-f9xb krb5kdc[24243](info): commencing operation
krb5kdc: starting...
W20260926 14:59:33.238979 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:33 dist-test-slave-f9xb krb5kdc[24243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:59:33.355610 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:38439
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:32779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:38439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:33.431013 24256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:33.431185 24256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:33.431212 24256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:33.432307 24256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:33.432343 24256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:33.432363 24256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:33.432377 24256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:33.432392 24256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:33.433840 24256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:32779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:38439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:38439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:33.434051 24256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:33.434226 24256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:33.438463 24261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:33.438472 24264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:33.438477 24262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:33.438711 24256 server_base.cc:1034] running on GCE node
I20260926 14:59:33.438925 24256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:33.439221 24256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:33.440356 24256 hybrid_clock.cc:648] HybridClock initialized: now 1790434773440342 us; error 22 us; skew 500 ppm
Sep 26 14:59:33 dist-test-slave-f9xb krb5kdc[24243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434773, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:33.442459 24256 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:59:33.442794 24256 webserver.cc:460] Webserver started at http://127.7.37.254:36739/ using document root <none> and password file <none>
I20260926 14:59:33.442972 24256 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:33.443017 24256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:33.443135 24256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:33.444052 24256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "65721288a6f44ce4ac5a2db1d38a3852"
format_stamp: "Formatted at 2026-09-26 14:59:33 on dist-test-slave-f9xb"
server_key: "d3d676258d6832a3e064eb86227e8496"
server_key_iv: "5938bdff09580cd7f8de322c42f93ec1"
server_key_version: "encryptionkey@0"
I20260926 14:59:33.444365 24256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "65721288a6f44ce4ac5a2db1d38a3852"
format_stamp: "Formatted at 2026-09-26 14:59:33 on dist-test-slave-f9xb"
server_key: "d3d676258d6832a3e064eb86227e8496"
server_key_iv: "5938bdff09580cd7f8de322c42f93ec1"
server_key_version: "encryptionkey@0"
I20260926 14:59:33.445456 24256 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:59:33.446017 24271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:33.446194 24256 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:33.446269 24256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "65721288a6f44ce4ac5a2db1d38a3852"
format_stamp: "Formatted at 2026-09-26 14:59:33 on dist-test-slave-f9xb"
server_key: "d3d676258d6832a3e064eb86227e8496"
server_key_iv: "5938bdff09580cd7f8de322c42f93ec1"
server_key_version: "encryptionkey@0"
I20260926 14:59:33.446324 24256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:33.458163 24256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:33.459641 24256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:33.459803 24256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:33.463016 24256 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:38439
I20260926 14:59:33.463053 24323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:38439 every 8 connection(s)
I20260926 14:59:33.463321 24256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:33.464138 24324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:33.466081 24324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852: Bootstrap starting.
I20260926 14:59:33.466586 24324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:33.466838 24324 log.cc:824] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:33.467487 24324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852: No bootstrap required, opened a new log
I20260926 14:59:33.469038 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 24256
I20260926 14:59:33.469154 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:33.469161 24324 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65721288a6f44ce4ac5a2db1d38a3852" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 38439 } }
I20260926 14:59:33.469317 24324 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:33.469364 7319 external_mini_cluster.cc:1428] Setting key f9fc5c0fa7421889ca4ec1ac0854aebc
I20260926 14:59:33.469358 24324 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65721288a6f44ce4ac5a2db1d38a3852, State: Initialized, Role: FOLLOWER
I20260926 14:59:33.469491 24324 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65721288a6f44ce4ac5a2db1d38a3852" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 38439 } }
I20260926 14:59:33.469569 24324 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:33.469597 24324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:33.469673 24324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:33.470271 24324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65721288a6f44ce4ac5a2db1d38a3852" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 38439 } }
I20260926 14:59:33.470369 24324 leader_election.cc:302] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65721288a6f44ce4ac5a2db1d38a3852; no voters:
I20260926 14:59:33.470549 24324 leader_election.cc:288] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:33.470588 24329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:33.470700 24329 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852 [term 1 LEADER]: Becoming Leader. State: Replica: 65721288a6f44ce4ac5a2db1d38a3852, State: Running, Role: LEADER
I20260926 14:59:33.470785 24324 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:33.470836 24329 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65721288a6f44ce4ac5a2db1d38a3852" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 38439 } }
I20260926 14:59:33.471179 24329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65721288a6f44ce4ac5a2db1d38a3852. Latest consensus state: current_term: 1 leader_uuid: "65721288a6f44ce4ac5a2db1d38a3852" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65721288a6f44ce4ac5a2db1d38a3852" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 38439 } } }
I20260926 14:59:33.471433 24329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:33.471354 24331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65721288a6f44ce4ac5a2db1d38a3852" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65721288a6f44ce4ac5a2db1d38a3852" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 38439 } } }
I20260926 14:59:33.471629 24331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:33.471897 24338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:33.472363 24338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:33.474622 24338 catalog_manager.cc:1347] Generated new cluster ID: 296359a0a66b4b5da66b52d90f2aa1e4
I20260926 14:59:33.474671 24338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 14:59:33 dist-test-slave-f9xb krb5kdc[24243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:59:33.480165 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:33.469728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44542 (local address 127.7.37.254:38439)
0926 14:59:33.469881 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:33.469889 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:33.469921 (+ 32us) server_negotiation.cc:407] Connection header received
0926 14:59:33.470233 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:33.470239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:33.470277 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:33.470388 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:33.471083 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.472419 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.473148 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.473713 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.473809 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:33.476446 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:33.476450 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:33.476450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:33.476460 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:33.478231 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:33.478604 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:33.478607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:33.478607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:33.478728 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:33.478936 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:33.478937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:33.478937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:33.479217 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:33.479364 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:33.479844 (+ 480us) server_negotiation.cc:299] Negotiation successful
0926 14:59:33.479972 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":87,"threads_started":1}
I20260926 14:59:33.480862 24338 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:33.481328 24338 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:33.485373 24338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852: Generated new TSK 0
I20260926 14:59:33.485509 24338 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
I20260926 14:59:33.510840 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:38439
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:32779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:33.592056 24350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:33.592247 24350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:33.592270 24350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:33.593372 24350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:33.593415 24350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:33.593441 24350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:59:33.594846 24350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:32779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:38439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:33.595055 24350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:33.595264 24350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:33.600615 24355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:33.600726 24350 server_base.cc:1034] running on GCE node
W20260926 14:59:33.600616 24358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:33.600601 24356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:33.601022 24350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:33.601257 24350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:33.602406 24350 hybrid_clock.cc:648] HybridClock initialized: now 1790434773602390 us; error 29 us; skew 500 ppm
Sep 26 14:59:33 dist-test-slave-f9xb krb5kdc[24243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434773, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:33.604657 24350 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:59:33.605036 24350 webserver.cc:460] Webserver started at http://127.7.37.193:37469/ using document root <none> and password file <none>
I20260926 14:59:33.605237 24350 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:33.605281 24350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:33.605397 24350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:33.606305 24350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "e0f69aa55c234877a21edaa3a66fde4b"
format_stamp: "Formatted at 2026-09-26 14:59:33 on dist-test-slave-f9xb"
server_key: "8ac7dc8ed7455d6021049f0a0af2ce9d"
server_key_iv: "c57c5e1eceaf1294844d1de57ae0b035"
server_key_version: "encryptionkey@0"
I20260926 14:59:33.606601 24350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "e0f69aa55c234877a21edaa3a66fde4b"
format_stamp: "Formatted at 2026-09-26 14:59:33 on dist-test-slave-f9xb"
server_key: "8ac7dc8ed7455d6021049f0a0af2ce9d"
server_key_iv: "c57c5e1eceaf1294844d1de57ae0b035"
server_key_version: "encryptionkey@0"
I20260926 14:59:33.607710 24350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:33.608346 24365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:33.608487 24350 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:33.608566 24350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "e0f69aa55c234877a21edaa3a66fde4b"
format_stamp: "Formatted at 2026-09-26 14:59:33 on dist-test-slave-f9xb"
server_key: "8ac7dc8ed7455d6021049f0a0af2ce9d"
server_key_iv: "c57c5e1eceaf1294844d1de57ae0b035"
server_key_version: "encryptionkey@0"
I20260926 14:59:33.608629 24350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:33.619978 24350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:33.621765 24350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:33.621882 24350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:33.622057 24350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:33.622300 24350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:59:33.622331 24350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:33.622356 24350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:59:33.622373 24350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:33.627530 24350 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:34749
I20260926 14:59:33.627562 24478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:34749 every 8 connection(s)
I20260926 14:59:33.627837 24350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
Sep 26 14:59:33 dist-test-slave-f9xb krb5kdc[24243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434773, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:59:33.634929 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 24350
I20260926 14:59:33.635030 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:59:33.635218 7319 external_mini_cluster.cc:1428] Setting key a0edf6a4fd6f774a0b2eb52020d8e4b7
I20260926 14:59:33.635499 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:33.628377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:45639 (local address 127.7.37.254:38439)
0926 14:59:33.628444 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:33.628447 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:33.628593 (+ 146us) server_negotiation.cc:407] Connection header received
0926 14:59:33.629026 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:33.629028 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:33.629041 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:33.629089 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:33.629918 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.630398 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.631287 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.631477 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.631528 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:33.633225 (+ 1697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:33.633227 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:33.633227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:33.633234 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:33.634301 (+ 1067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:33.634617 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:33.634618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:33.634619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:33.634668 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:33.634858 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:33.634858 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:33.634859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:33.634955 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:33.635011 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:33.635338 (+ 327us) server_negotiation.cc:299] Negotiation successful
0926 14:59:33.635415 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 14:59:33.635645 24481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:33.628347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:38439 (local address 127.7.37.193:45639)
0926 14:59:33.628490 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:33.628524 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:33.628939 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:33.629150 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:33.629153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:33.629658 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:33.629831 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:33.629835 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.630469 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.630470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:33.631211 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:33.631213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.631538 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.631539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:33.631654 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:33.632000 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:59:33.632007 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:33.633148 (+ 1141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:33.634367 (+ 1219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:33.634373 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:33.634375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:33.634558 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:33.634719 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:33.634720 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:33.634720 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:33.634805 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:33.635026 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:33.635033 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:33.635203 (+ 170us) client_negotiation.cc:769] Sending connection context
0926 14:59:33.635317 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 14:59:33.635455 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260926 14:59:33.636021 24479 heartbeater.cc:347] Connected to a master server at 127.7.37.254:38439
I20260926 14:59:33.636097 24479 heartbeater.cc:464] Registering TS with master...
I20260926 14:59:33.636238 24479 heartbeater.cc:511] Master 127.7.37.254:38439 requested a full tablet report, sending...
I20260926 14:59:33.636636 24288 ts_manager.cc:194] Registered new tserver with Master: e0f69aa55c234877a21edaa3a66fde4b (127.7.37.193:34749)
I20260926 14:59:33.637179 24288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:45639
WARNING: no policy specified for kudu/127.7.37.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
I20260926 14:59:33.656293 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:38439
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:32779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:33.731063 24484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:33.731235 24484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:33.731266 24484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:33.732362 24484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:33.732399 24484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:33.732434 24484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:59:33.733812 24484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:32779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:38439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:33.733994 24484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:33.734154 24484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:33.738601 24490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:33.738642 24484 server_base.cc:1034] running on GCE node
W20260926 14:59:33.738592 24492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:33.738575 24489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:33.738890 24484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:33.739115 24484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:33.740269 24484 hybrid_clock.cc:648] HybridClock initialized: now 1790434773740255 us; error 30 us; skew 500 ppm
Sep 26 14:59:33 dist-test-slave-f9xb krb5kdc[24243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434773, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:33.742512 24484 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:59:33.742882 24484 webserver.cc:460] Webserver started at http://127.7.37.194:43471/ using document root <none> and password file <none>
I20260926 14:59:33.743063 24484 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:33.743103 24484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:33.743208 24484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:33.744351 24484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "2c9ce08a56ec4d129fb5c6892404650a"
format_stamp: "Formatted at 2026-09-26 14:59:33 on dist-test-slave-f9xb"
server_key: "91fce63aa3d9fd25a36bfc0b06f7ed25"
server_key_iv: "c3068d0ba6a7f7ee33106034e2b2006c"
server_key_version: "encryptionkey@0"
I20260926 14:59:33.744695 24484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "2c9ce08a56ec4d129fb5c6892404650a"
format_stamp: "Formatted at 2026-09-26 14:59:33 on dist-test-slave-f9xb"
server_key: "91fce63aa3d9fd25a36bfc0b06f7ed25"
server_key_iv: "c3068d0ba6a7f7ee33106034e2b2006c"
server_key_version: "encryptionkey@0"
I20260926 14:59:33.745793 24484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:33.746376 24499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:33.746510 24484 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:33.746587 24484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "2c9ce08a56ec4d129fb5c6892404650a"
format_stamp: "Formatted at 2026-09-26 14:59:33 on dist-test-slave-f9xb"
server_key: "91fce63aa3d9fd25a36bfc0b06f7ed25"
server_key_iv: "c3068d0ba6a7f7ee33106034e2b2006c"
server_key_version: "encryptionkey@0"
I20260926 14:59:33.746637 24484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:33.759958 24484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:33.761504 24484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:33.761598 24484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:33.761749 24484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:33.761951 24484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:59:33.761978 24484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:33.761996 24484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:59:33.762012 24484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:33.766961 24484 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:40571
I20260926 14:59:33.767024 24612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:40571 every 8 connection(s)
I20260926 14:59:33.767263 24484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:59:33.769647 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 24484
I20260926 14:59:33.769750 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:59:33.769928 7319 external_mini_cluster.cc:1428] Setting key bbd6cc1089f3d70f8941d6212cddc70f
Sep 26 14:59:33 dist-test-slave-f9xb krb5kdc[24243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434773, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:59:33.776176 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:33.767992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:54663 (local address 127.7.37.254:38439)
0926 14:59:33.768051 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:33.768053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:33.768245 (+ 192us) server_negotiation.cc:407] Connection header received
0926 14:59:33.768624 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:33.768625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:33.768636 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:33.768685 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:33.769468 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.770082 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.771162 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.771370 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.771423 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:33.773753 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:33.773755 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:33.773755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:33.773762 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:33.774964 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:33.775312 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:33.775313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:33.775314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:33.775369 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:33.775571 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:33.775572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:33.775572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:33.775681 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:33.775776 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:33.776007 (+ 231us) server_negotiation.cc:299] Negotiation successful
0926 14:59:33.776091 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 14:59:33.776312 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:33.767969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:38439 (local address 127.7.37.194:54663)
0926 14:59:33.768141 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:33.768176 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:33.768553 (+ 377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:33.768737 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:33.768740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:33.769241 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:33.769401 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:33.769405 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.770174 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.770175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:33.771088 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:33.771091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.771432 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.771433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:33.771561 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:33.772128 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:59:33.772137 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:33.773662 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:33.775034 (+ 1372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:33.775040 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:33.775042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:33.775248 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:33.775421 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:33.775422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:33.775422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:33.775519 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:33.775770 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:33.775775 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:33.775898 (+ 123us) client_negotiation.cc:769] Sending connection context
0926 14:59:33.775980 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 14:59:33.776113 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":116,"threads_started":1}
I20260926 14:59:33.776693 24613 heartbeater.cc:347] Connected to a master server at 127.7.37.254:38439
I20260926 14:59:33.776789 24613 heartbeater.cc:464] Registering TS with master...
I20260926 14:59:33.776955 24613 heartbeater.cc:511] Master 127.7.37.254:38439 requested a full tablet report, sending...
I20260926 14:59:33.777305 24288 ts_manager.cc:194] Registered new tserver with Master: 2c9ce08a56ec4d129fb5c6892404650a (127.7.37.194:40571)
I20260926 14:59:33.777797 24288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:54663
WARNING: no policy specified for kudu/127.7.37.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
I20260926 14:59:33.792390 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:38439
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:32779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:33.867027 24618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:33.867203 24618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:33.867224 24618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:33.868363 24618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:33.868402 24618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:33.868432 24618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:59:33.869810 24618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:32779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:38439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:33.870015 24618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:33.870213 24618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:33.874626 24623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:33.874677 24626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:33.874672 24624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:33.874672 24618 server_base.cc:1034] running on GCE node
I20260926 14:59:33.874994 24618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:33.875203 24618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:33.876336 24618 hybrid_clock.cc:648] HybridClock initialized: now 1790434773876323 us; error 27 us; skew 500 ppm
Sep 26 14:59:33 dist-test-slave-f9xb krb5kdc[24243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434773, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:33.878460 24618 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:59:33.878841 24618 webserver.cc:460] Webserver started at http://127.7.37.195:44727/ using document root <none> and password file <none>
I20260926 14:59:33.879019 24618 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:33.879061 24618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:33.879187 24618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:33.880092 24618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "744f1dfa11514cc69f16c504226eedd3"
format_stamp: "Formatted at 2026-09-26 14:59:33 on dist-test-slave-f9xb"
server_key: "f31781639668d32c2c385190ee8f30b8"
server_key_iv: "211af106a2396275f0299bcd03c26511"
server_key_version: "encryptionkey@0"
I20260926 14:59:33.880388 24618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "744f1dfa11514cc69f16c504226eedd3"
format_stamp: "Formatted at 2026-09-26 14:59:33 on dist-test-slave-f9xb"
server_key: "f31781639668d32c2c385190ee8f30b8"
server_key_iv: "211af106a2396275f0299bcd03c26511"
server_key_version: "encryptionkey@0"
I20260926 14:59:33.881445 24618 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:59:33.882019 24633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:33.882167 24618 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:33.882239 24618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "744f1dfa11514cc69f16c504226eedd3"
format_stamp: "Formatted at 2026-09-26 14:59:33 on dist-test-slave-f9xb"
server_key: "f31781639668d32c2c385190ee8f30b8"
server_key_iv: "211af106a2396275f0299bcd03c26511"
server_key_version: "encryptionkey@0"
I20260926 14:59:33.882295 24618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:33.895383 24618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:33.896927 24618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:33.897042 24618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:33.897215 24618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:33.897450 24618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:59:33.897481 24618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:33.897506 24618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:59:33.897528 24618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:33.902452 24618 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:38921
I20260926 14:59:33.902506 24746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:38921 every 8 connection(s)
I20260926 14:59:33.902736 24618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:59:33.905911 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 24618
I20260926 14:59:33.905998 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:59:33.906164 7319 external_mini_cluster.cc:1428] Setting key d93dab49bc42f90606127bbac4a51a92
Sep 26 14:59:33 dist-test-slave-f9xb krb5kdc[24243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434773, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:59:33.910310 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:33.903311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:38067 (local address 127.7.37.254:38439)
0926 14:59:33.903372 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:33.903374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:33.903522 (+ 148us) server_negotiation.cc:407] Connection header received
0926 14:59:33.903951 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:33.903953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:33.903964 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:33.904011 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:33.904824 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.905295 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.906240 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.906469 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.906535 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:33.908391 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:33.908392 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:33.908393 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:33.908399 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:33.909372 (+ 973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:33.909679 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:33.909679 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:33.909680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:33.909720 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:33.909889 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:33.909889 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:33.909890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:33.909977 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:33.910019 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:33.910200 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 14:59:33.910252 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 14:59:33.910434 24749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:33.903260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:38439 (local address 127.7.37.195:38067)
0926 14:59:33.903417 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:33.903454 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:33.903863 (+ 409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:33.904066 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:33.904070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:33.904577 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:33.904735 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:33.904739 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.905366 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.905367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:33.906160 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:33.906162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.906540 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.906541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:33.906672 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:33.907106 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:59:33.907113 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:33.908322 (+ 1209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:33.909437 (+ 1115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:33.909442 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:33.909443 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:33.909628 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:33.909761 (+ 133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:33.909762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:33.909762 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:33.909849 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:33.910015 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:33.910017 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:33.910107 (+ 90us) client_negotiation.cc:769] Sending connection context
0926 14:59:33.910180 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 14:59:33.910280 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
I20260926 14:59:33.910818 24747 heartbeater.cc:347] Connected to a master server at 127.7.37.254:38439
I20260926 14:59:33.910892 24747 heartbeater.cc:464] Registering TS with master...
I20260926 14:59:33.911067 24747 heartbeater.cc:511] Master 127.7.37.254:38439 requested a full tablet report, sending...
I20260926 14:59:33.911381 24288 ts_manager.cc:194] Registered new tserver with Master: 744f1dfa11514cc69f16c504226eedd3 (127.7.37.195:38921)
I20260926 14:59:33.911720 24288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:38067
I20260926 14:59:33.917490 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 14:59:33.923282 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:33.918616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44544 (local address 127.7.37.254:38439)
0926 14:59:33.918660 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:33.918662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:33.918758 (+ 96us) server_negotiation.cc:407] Connection header received
0926 14:59:33.918792 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:33.918792 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:33.918799 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:33.918838 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:33.919307 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.919729 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.920397 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.920561 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.920601 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:33.921532 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:33.921533 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:33.921533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:33.921538 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:33.922517 (+ 979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:33.922780 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:33.922780 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:33.922780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:33.922817 (+ 37us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:33.922918 (+ 101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:33.922918 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:33.922918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:33.923030 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:33.923092 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:33.923177 (+ 85us) server_negotiation.cc:299] Negotiation successful
0926 14:59:33.923225 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 14:59:33.924594 24288 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:59:33.925958 24288 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:59:33.931319 24757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:33.927653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:40571 (local address 127.0.0.1:56022)
0926 14:59:33.928239 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:33.928248 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:33.928298 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:33.928520 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:33.928521 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:33.928522 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:33.928569 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:33.928570 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.929394 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.929396 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:33.930224 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:33.930227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.931055 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.931055 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:33.931125 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:33.931137 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 14:59:33.931204 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 14:59:33.931251 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":33,"thread_start_us":70,"threads_started":1}
I20260926 14:59:33.931339 24760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:33.928070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56022 (local address 127.7.37.194:40571)
0926 14:59:33.928223 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:33.928227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:33.928287 (+ 60us) server_negotiation.cc:407] Connection header received
0926 14:59:33.928355 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:33.928357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:33.928371 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:33.928447 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:33.928639 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.929305 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.930303 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.930978 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.931047 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:33.931097 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:33.931222 (+ 125us) server_negotiation.cc:299] Negotiation successful
0926 14:59:33.931259 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":75,"threads_started":1}
I20260926 14:59:33.931651 24755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:33.927623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:38921 (local address 127.0.0.1:51386)
0926 14:59:33.927795 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:33.927828 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:33.927886 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:33.928057 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:33.928060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:33.928062 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:33.928118 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:33.928121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.929330 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.929331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:33.930717 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:33.930719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.931454 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.931455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:33.931513 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:33.931516 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:59:33.931557 (+ 41us) client_negotiation.cc:240] Negotiation successful
0926 14:59:33.931590 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":39,"thread_start_us":62,"threads_started":1}
I20260926 14:59:33.931721 24759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:33.927885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36466 (local address 127.7.37.193:34749)
0926 14:59:33.928042 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:33.928046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:33.928054 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:59:33.928086 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:33.928088 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:33.928102 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:33.928198 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:33.928382 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.929384 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.930687 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.931351 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.931414 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:33.931467 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:33.931616 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 14:59:33.931651 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260926 14:59:33.931973 24758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:33.927733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51386 (local address 127.7.37.195:38921)
0926 14:59:33.927877 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:33.927881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:33.927890 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:59:33.927956 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:33.927958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:33.927971 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:33.928048 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:33.928197 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.929230 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.930777 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.931310 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.931366 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:33.931420 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:33.931893 (+ 473us) server_negotiation.cc:299] Negotiation successful
0926 14:59:33.931927 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260926 14:59:33.931734 24756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:33.927623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:34749 (local address 127.0.0.1:36466)
0926 14:59:33.927795 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:33.927828 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:33.927886 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:33.928274 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:33.928274 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:33.928276 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:33.928310 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:33.928311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.929466 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.929467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:33.930587 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:33.930589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:33.931499 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:33.931500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:33.931546 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:33.931549 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:59:33.931594 (+ 45us) client_negotiation.cc:240] Negotiation successful
0926 14:59:33.931633 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":5,"thread_start_us":84,"threads_started":1}
I20260926 14:59:33.932221 24547 tablet_service.cc:1467] Processing CreateTablet for tablet 9140b299011640eba2a1127f9bb7a0aa (DEFAULT_TABLE table=test-table [id=28f79d8c990242a593f2b0ba240dd791]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:59:33.932531 24547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9140b299011640eba2a1127f9bb7a0aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:33.932547 24681 tablet_service.cc:1467] Processing CreateTablet for tablet 9140b299011640eba2a1127f9bb7a0aa (DEFAULT_TABLE table=test-table [id=28f79d8c990242a593f2b0ba240dd791]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:59:33.932816 24681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9140b299011640eba2a1127f9bb7a0aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:33.932813 24413 tablet_service.cc:1467] Processing CreateTablet for tablet 9140b299011640eba2a1127f9bb7a0aa (DEFAULT_TABLE table=test-table [id=28f79d8c990242a593f2b0ba240dd791]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:59:33.933075 24413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9140b299011640eba2a1127f9bb7a0aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:33.935007 24761 tablet_bootstrap.cc:492] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a: Bootstrap starting.
I20260926 14:59:33.935652 24761 tablet_bootstrap.cc:654] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:33.935631 24762 tablet_bootstrap.cc:492] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3: Bootstrap starting.
I20260926 14:59:33.936057 24763 tablet_bootstrap.cc:492] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b: Bootstrap starting.
I20260926 14:59:33.936250 24762 tablet_bootstrap.cc:654] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:33.936230 24761 log.cc:824] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:33.936650 24763 tablet_bootstrap.cc:654] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:33.936728 24762 log.cc:824] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:33.936954 24761 tablet_bootstrap.cc:492] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a: No bootstrap required, opened a new log
I20260926 14:59:33.937141 24763 log.cc:824] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:33.937172 24761 ts_tablet_manager.cc:1397] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:59:33.937457 24762 tablet_bootstrap.cc:492] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3: No bootstrap required, opened a new log
I20260926 14:59:33.937521 24762 ts_tablet_manager.cc:1397] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 14:59:33.937793 24763 tablet_bootstrap.cc:492] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b: No bootstrap required, opened a new log
I20260926 14:59:33.937847 24763 ts_tablet_manager.cc:1397] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:59:33.938992 24761 raft_consensus.cc:348] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0f69aa55c234877a21edaa3a66fde4b" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34749 } } peers { permanent_uuid: "744f1dfa11514cc69f16c504226eedd3" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38921 } } peers { permanent_uuid: "2c9ce08a56ec4d129fb5c6892404650a" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 40571 } }
I20260926 14:59:33.939157 24761 raft_consensus.cc:374] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:33.939086 24763 raft_consensus.cc:348] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0f69aa55c234877a21edaa3a66fde4b" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34749 } } peers { permanent_uuid: "744f1dfa11514cc69f16c504226eedd3" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38921 } } peers { permanent_uuid: "2c9ce08a56ec4d129fb5c6892404650a" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 40571 } }
I20260926 14:59:33.939193 24761 raft_consensus.cc:729] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9ce08a56ec4d129fb5c6892404650a, State: Initialized, Role: FOLLOWER
I20260926 14:59:33.939237 24763 raft_consensus.cc:374] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:33.939298 24763 raft_consensus.cc:729] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0f69aa55c234877a21edaa3a66fde4b, State: Initialized, Role: FOLLOWER
I20260926 14:59:33.939314 24761 consensus_queue.cc:260] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0f69aa55c234877a21edaa3a66fde4b" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34749 } } peers { permanent_uuid: "744f1dfa11514cc69f16c504226eedd3" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38921 } } peers { permanent_uuid: "2c9ce08a56ec4d129fb5c6892404650a" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 40571 } }
I20260926 14:59:33.939265 24762 raft_consensus.cc:348] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0f69aa55c234877a21edaa3a66fde4b" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34749 } } peers { permanent_uuid: "744f1dfa11514cc69f16c504226eedd3" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38921 } } peers { permanent_uuid: "2c9ce08a56ec4d129fb5c6892404650a" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 40571 } }
I20260926 14:59:33.939397 24762 raft_consensus.cc:374] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:33.939409 24763 consensus_queue.cc:260] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0f69aa55c234877a21edaa3a66fde4b" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34749 } } peers { permanent_uuid: "744f1dfa11514cc69f16c504226eedd3" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38921 } } peers { permanent_uuid: "2c9ce08a56ec4d129fb5c6892404650a" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 40571 } }
I20260926 14:59:33.939458 24762 raft_consensus.cc:729] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 744f1dfa11514cc69f16c504226eedd3, State: Initialized, Role: FOLLOWER
I20260926 14:59:33.939590 24761 ts_tablet_manager.cc:1428] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:59:33.939559 24762 consensus_queue.cc:260] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0f69aa55c234877a21edaa3a66fde4b" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34749 } } peers { permanent_uuid: "744f1dfa11514cc69f16c504226eedd3" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38921 } } peers { permanent_uuid: "2c9ce08a56ec4d129fb5c6892404650a" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 40571 } }
I20260926 14:59:33.939656 24479 heartbeater.cc:503] Master 127.7.37.254:38439 was elected leader, sending a full tablet report...
I20260926 14:59:33.939656 24763 ts_tablet_manager.cc:1428] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:59:33.939790 24613 heartbeater.cc:503] Master 127.7.37.254:38439 was elected leader, sending a full tablet report...
I20260926 14:59:33.939803 24762 ts_tablet_manager.cc:1428] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 14:59:33.939908 24747 heartbeater.cc:503] Master 127.7.37.254:38439 was elected leader, sending a full tablet report...
I20260926 14:59:34.003139 24768 raft_consensus.cc:482] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:59:34.003239 24768 raft_consensus.cc:504] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0f69aa55c234877a21edaa3a66fde4b" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34749 } } peers { permanent_uuid: "744f1dfa11514cc69f16c504226eedd3" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38921 } } peers { permanent_uuid: "2c9ce08a56ec4d129fb5c6892404650a" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 40571 } }
I20260926 14:59:34.003557 24768 leader_election.cc:288] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 744f1dfa11514cc69f16c504226eedd3 (127.7.37.195:38921), 2c9ce08a56ec4d129fb5c6892404650a (127.7.37.194:40571)
I20260926 14:59:34.006007 24481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.003681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:38921 (local address 127.7.37.193:36475)
0926 14:59:34.003737 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:34.003762 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:34.003800 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:34.003916 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:34.003917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:34.003919 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:34.003981 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.003983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.004547 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.004548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.005287 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.005289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.005793 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.005794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.005859 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.005862 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:59:34.005904 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 14:59:34.005939 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":21}
I20260926 14:59:34.006003 24758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.003767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:36475 (local address 127.7.37.195:38921)
0926 14:59:34.003822 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:34.003824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:34.003832 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:59:34.003854 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:34.003854 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:34.003862 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:34.003909 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:34.004030 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.004485 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.005360 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.005733 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.005788 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.005796 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:34.005913 (+ 117us) server_negotiation.cc:299] Negotiation successful
0926 14:59:34.005943 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 14:59:34.006117 24770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.003696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:40571 (local address 127.7.37.193:49849)
0926 14:59:34.003861 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:34.003871 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:34.003923 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:34.004040 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:34.004041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:34.004042 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:34.004104 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.004105 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.004659 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.004660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.005382 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.005384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.005896 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.005898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.005960 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.005963 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:59:34.006006 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 14:59:34.006040 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":97,"threads_started":1}
I20260926 14:59:34.006107 24760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.003773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:49849 (local address 127.7.37.194:40571)
0926 14:59:34.003836 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:34.003837 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:34.003921 (+ 84us) server_negotiation.cc:407] Connection header received
0926 14:59:34.003979 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:34.003979 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:34.003988 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:34.004031 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:34.004153 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.004592 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.005453 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.005828 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.005888 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.005896 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:34.006014 (+ 118us) server_negotiation.cc:299] Negotiation successful
0926 14:59:34.006047 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 14:59:34.006445 24567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9140b299011640eba2a1127f9bb7a0aa" candidate_uuid: "e0f69aa55c234877a21edaa3a66fde4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9ce08a56ec4d129fb5c6892404650a" is_pre_election: true
I20260926 14:59:34.006480 24701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9140b299011640eba2a1127f9bb7a0aa" candidate_uuid: "e0f69aa55c234877a21edaa3a66fde4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "744f1dfa11514cc69f16c504226eedd3" is_pre_election: true
I20260926 14:59:34.006567 24567 raft_consensus.cc:2454] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0f69aa55c234877a21edaa3a66fde4b in term 0.
I20260926 14:59:34.006584 24701 raft_consensus.cc:2454] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0f69aa55c234877a21edaa3a66fde4b in term 0.
I20260926 14:59:34.006760 24367 leader_election.cc:302] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 744f1dfa11514cc69f16c504226eedd3, e0f69aa55c234877a21edaa3a66fde4b; no voters:
I20260926 14:59:34.006863 24768 raft_consensus.cc:2781] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:59:34.006912 24768 raft_consensus.cc:482] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:59:34.006933 24768 raft_consensus.cc:3037] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:34.007524 24768 raft_consensus.cc:504] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0f69aa55c234877a21edaa3a66fde4b" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34749 } } peers { permanent_uuid: "744f1dfa11514cc69f16c504226eedd3" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38921 } } peers { permanent_uuid: "2c9ce08a56ec4d129fb5c6892404650a" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 40571 } }
I20260926 14:59:34.007622 24768 leader_election.cc:288] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b [CANDIDATE]: Term 1 election: Requested vote from peers 744f1dfa11514cc69f16c504226eedd3 (127.7.37.195:38921), 2c9ce08a56ec4d129fb5c6892404650a (127.7.37.194:40571)
I20260926 14:59:34.007783 24567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9140b299011640eba2a1127f9bb7a0aa" candidate_uuid: "e0f69aa55c234877a21edaa3a66fde4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9ce08a56ec4d129fb5c6892404650a"
I20260926 14:59:34.007807 24701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9140b299011640eba2a1127f9bb7a0aa" candidate_uuid: "e0f69aa55c234877a21edaa3a66fde4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "744f1dfa11514cc69f16c504226eedd3"
I20260926 14:59:34.007846 24567 raft_consensus.cc:3037] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:34.007865 24701 raft_consensus.cc:3037] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:34.008457 24567 raft_consensus.cc:2454] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0f69aa55c234877a21edaa3a66fde4b in term 1.
I20260926 14:59:34.008472 24701 raft_consensus.cc:2454] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0f69aa55c234877a21edaa3a66fde4b in term 1.
I20260926 14:59:34.008592 24367 leader_election.cc:302] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 744f1dfa11514cc69f16c504226eedd3, e0f69aa55c234877a21edaa3a66fde4b; no voters:
I20260926 14:59:34.008682 24768 raft_consensus.cc:2781] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:34.008795 24768 raft_consensus.cc:686] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b [term 1 LEADER]: Becoming Leader. State: Replica: e0f69aa55c234877a21edaa3a66fde4b, State: Running, Role: LEADER
I20260926 14:59:34.008857 24768 consensus_queue.cc:237] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0f69aa55c234877a21edaa3a66fde4b" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34749 } } peers { permanent_uuid: "744f1dfa11514cc69f16c504226eedd3" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38921 } } peers { permanent_uuid: "2c9ce08a56ec4d129fb5c6892404650a" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 40571 } }
I20260926 14:59:34.009528 24287 catalog_manager.cc:5495] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b reported cstate change: term changed from 0 to 1, leader changed from <none> to e0f69aa55c234877a21edaa3a66fde4b (127.7.37.193). New cstate: current_term: 1 leader_uuid: "e0f69aa55c234877a21edaa3a66fde4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0f69aa55c234877a21edaa3a66fde4b" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "744f1dfa11514cc69f16c504226eedd3" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c9ce08a56ec4d129fb5c6892404650a" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 40571 } health_report { overall_health: UNKNOWN } } }
W20260926 14:59:34.017763 24614 tablet.cc:2369] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:59:34.023236 24759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.021236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36476 (local address 127.7.37.193:34749)
0926 14:59:34.021280 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:34.021282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:34.021287 (+ 5us) server_negotiation.cc:407] Connection header received
0926 14:59:34.021310 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:34.021310 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:34.021323 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:34.021376 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:59:34.021490 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.021907 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.022599 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.022777 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.022827 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.022966 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:59:34.023045 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:59:34.023109 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:34.023144 (+ 35us) server_negotiation.cc:299] Negotiation successful
0926 14:59:34.023162 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 14:59:34.024645 24701 raft_consensus.cc:1261] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3 [term 1 FOLLOWER]: Refusing update from remote peer e0f69aa55c234877a21edaa3a66fde4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:59:34.024654 24567 raft_consensus.cc:1261] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a [term 1 FOLLOWER]: Refusing update from remote peer e0f69aa55c234877a21edaa3a66fde4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:59:34.024917 24768 consensus_queue.cc:1035] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9ce08a56ec4d129fb5c6892404650a" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 40571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:59:34.025029 24771 consensus_queue.cc:1035] T 9140b299011640eba2a1127f9bb7a0aa P e0f69aa55c234877a21edaa3a66fde4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "744f1dfa11514cc69f16c504226eedd3" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 38921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:59:34.027000 24774 mvcc.cc:205] Tried to move back new op lower bound from 7333620834402930688 to 7333620834340610048. Current Snapshot: MvccSnapshot[applied={T|T < 7333620834402930688}]
I20260926 14:59:34.027141 24776 mvcc.cc:205] Tried to move back new op lower bound from 7333620834402930688 to 7333620834340610048. Current Snapshot: MvccSnapshot[applied={T|T < 7333620834402930688}]
I20260926 14:59:34.028057 24775 mvcc.cc:205] Tried to move back new op lower bound from 7333620834402930688 to 7333620834340610048. Current Snapshot: MvccSnapshot[applied={T|T < 7333620834402930688 or (T in {7333620834402930688})}]
I20260926 14:59:34.029103 24286 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:59:34.029183 24286 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:59:34.029801 24286 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 65721288a6f44ce4ac5a2db1d38a3852: Sending DeleteTablet for 3 replicas of tablet 9140b299011640eba2a1127f9bb7a0aa
I20260926 14:59:34.030071 24681 tablet_service.cc:1514] Processing DeleteTablet for tablet 9140b299011640eba2a1127f9bb7a0aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:59:34 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:51386
I20260926 14:59:34.030124 24547 tablet_service.cc:1514] Processing DeleteTablet for tablet 9140b299011640eba2a1127f9bb7a0aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:59:34 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:56022
I20260926 14:59:34.030128 24413 tablet_service.cc:1514] Processing DeleteTablet for tablet 9140b299011640eba2a1127f9bb7a0aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:59:34 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:36466
I20260926 14:59:34.030325 24783 tablet_replica.cc:330] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3: stopping tablet replica
I20260926 14:59:34.030418 24783 raft_consensus.cc:2229] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:59:34.030508 24783 raft_consensus.cc:2258] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:59:34.030570 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 24350
I20260926 14:59:34.031159 24784 tablet_replica.cc:330] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a: stopping tablet replica
I20260926 14:59:34.031282 24784 raft_consensus.cc:2229] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:59:34.031373 24784 raft_consensus.cc:2258] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:59:34.031821 24784 ts_tablet_manager.cc:1905] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:59:34.033316 24783 ts_tablet_manager.cc:1905] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:59:34.033466 24784 ts_tablet_manager.cc:1918] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:59:34.033524 24784 log.cc:1196] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/9140b299011640eba2a1127f9bb7a0aa
I20260926 14:59:34.033763 24784 ts_tablet_manager.cc:1939] T 9140b299011640eba2a1127f9bb7a0aa P 2c9ce08a56ec4d129fb5c6892404650a: Deleting consensus metadata
I20260926 14:59:34.034080 24272 catalog_manager.cc:4841] TS 2c9ce08a56ec4d129fb5c6892404650a (127.7.37.194:40571): tablet 9140b299011640eba2a1127f9bb7a0aa (table test-table [id=28f79d8c990242a593f2b0ba240dd791]) successfully deleted
I20260926 14:59:34.034835 24783 ts_tablet_manager.cc:1918] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:59:34.034896 24783 log.cc:1196] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/9140b299011640eba2a1127f9bb7a0aa
I20260926 14:59:34.035116 24783 ts_tablet_manager.cc:1939] T 9140b299011640eba2a1127f9bb7a0aa P 744f1dfa11514cc69f16c504226eedd3: Deleting consensus metadata
I20260926 14:59:34.035491 24273 catalog_manager.cc:4841] TS 744f1dfa11514cc69f16c504226eedd3 (127.7.37.195:38921): tablet 9140b299011640eba2a1127f9bb7a0aa (table test-table [id=28f79d8c990242a593f2b0ba240dd791]) successfully deleted
W20260926 14:59:34.036248 24275 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.37.193:34749 (error 108)
I20260926 14:59:34.036320 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 24484
I20260926 14:59:34.037135 24756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.037037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:34749 (local address 127.0.0.1:36488)
0926 14:59:34.037079 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:34.037111 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.193:34749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 14:59:34.037227 24275 catalog_manager.cc:4584] TS e0f69aa55c234877a21edaa3a66fde4b (127.7.37.193:34749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9140b299011640eba2a1127f9bb7a0aa: Network error: Client connection negotiation failed: client connection to 127.7.37.193:34749: connect: Connection refused (error 111)
I20260926 14:59:34.040540 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 24618
I20260926 14:59:34.044467 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 24256
2026-09-26T14:59:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1749 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-26T14:59:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:34Z Disabled control of system clock
I20260926 14:59:34.068284 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:45023
--webserver_interface=127.7.37.254
--webserver_port=0
--builtin_ntp_servers=127.7.37.212:42907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:45023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 14:59:34.143257 24790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:34.143442 24790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:34.143471 24790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:34.144572 24790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:34.144610 24790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:34.144626 24790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:34.144642 24790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:34.144657 24790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:34.145918 24790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:45023
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:45023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:34.146154 24790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:34.146389 24790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:34.150746 24798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:34.150781 24795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:34.150761 24796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:34.150949 24790 server_base.cc:1034] running on GCE node
I20260926 14:59:34.151188 24790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:34.151427 24790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:34.152570 24790 hybrid_clock.cc:648] HybridClock initialized: now 1790434774152555 us; error 26 us; skew 500 ppm
I20260926 14:59:34.157070 24790 webserver.cc:460] Webserver started at http://127.7.37.254:38705/ using document root <none> and password file <none>
I20260926 14:59:34.157299 24790 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:34.157348 24790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:34.157466 24790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:34.158448 24790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "9c2d1cfd7a9e43b7a89479fc2ab571c1"
format_stamp: "Formatted at 2026-09-26 14:59:34 on dist-test-slave-f9xb"
server_key: "6e6dfbef3482f06cb4387ce83dedc3fd"
server_key_iv: "339db4aba8fc839331af69ef01d3f5ba"
server_key_version: "encryptionkey@0"
I20260926 14:59:34.158751 24790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "9c2d1cfd7a9e43b7a89479fc2ab571c1"
format_stamp: "Formatted at 2026-09-26 14:59:34 on dist-test-slave-f9xb"
server_key: "6e6dfbef3482f06cb4387ce83dedc3fd"
server_key_iv: "339db4aba8fc839331af69ef01d3f5ba"
server_key_version: "encryptionkey@0"
I20260926 14:59:34.159895 24790 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:34.160600 24804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:34.160751 24790 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:34.160826 24790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "9c2d1cfd7a9e43b7a89479fc2ab571c1"
format_stamp: "Formatted at 2026-09-26 14:59:34 on dist-test-slave-f9xb"
server_key: "6e6dfbef3482f06cb4387ce83dedc3fd"
server_key_iv: "339db4aba8fc839331af69ef01d3f5ba"
server_key_version: "encryptionkey@0"
I20260926 14:59:34.160893 24790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:34.181313 24790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:34.181550 24790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:34.181663 24790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:34.185016 24790 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:45023
I20260926 14:59:34.185047 24856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:45023 every 8 connection(s)
I20260926 14:59:34.185324 24790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:34.186121 24857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:34.188181 24857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1: Bootstrap starting.
I20260926 14:59:34.188690 24857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:34.188937 24857 log.cc:824] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:34.189505 24857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1: No bootstrap required, opened a new log
I20260926 14:59:34.190982 24857 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2d1cfd7a9e43b7a89479fc2ab571c1" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 45023 } }
I20260926 14:59:34.191102 24857 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:34.191123 24857 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2d1cfd7a9e43b7a89479fc2ab571c1, State: Initialized, Role: FOLLOWER
I20260926 14:59:34.191246 24857 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2d1cfd7a9e43b7a89479fc2ab571c1" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 45023 } }
I20260926 14:59:34.191311 24857 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:34.191342 24857 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:34.191388 24857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:34.191699 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 24790
I20260926 14:59:34.191830 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:34.192021 7319 external_mini_cluster.cc:1428] Setting key 4447d1c51ea8da469e1256c217c7e9d7
I20260926 14:59:34.192013 24857 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2d1cfd7a9e43b7a89479fc2ab571c1" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 45023 } }
I20260926 14:59:34.192099 24857 leader_election.cc:302] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c2d1cfd7a9e43b7a89479fc2ab571c1; no voters:
I20260926 14:59:34.192252 24857 leader_election.cc:288] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:34.192279 24860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:34.192387 24860 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2d1cfd7a9e43b7a89479fc2ab571c1, State: Running, Role: LEADER
I20260926 14:59:34.192452 24860 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2d1cfd7a9e43b7a89479fc2ab571c1" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 45023 } }
I20260926 14:59:34.192479 24857 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:34.192821 24862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c2d1cfd7a9e43b7a89479fc2ab571c1. Latest consensus state: current_term: 1 leader_uuid: "9c2d1cfd7a9e43b7a89479fc2ab571c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2d1cfd7a9e43b7a89479fc2ab571c1" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 45023 } } }
I20260926 14:59:34.192898 24862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:34.193038 24861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c2d1cfd7a9e43b7a89479fc2ab571c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2d1cfd7a9e43b7a89479fc2ab571c1" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 45023 } } }
I20260926 14:59:34.193095 24861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:34.193463 24868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:34.193958 24868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:34.195235 24868 catalog_manager.cc:1347] Generated new cluster ID: deb043f5d7fb431d9ea7dd2eaf44692f
I20260926 14:59:34.195281 24868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:59:34.198290 24864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.192442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38108 (local address 127.7.37.254:45023)
0926 14:59:34.192822 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:34.192830 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:34.193312 (+ 482us) server_negotiation.cc:407] Connection header received
0926 14:59:34.193532 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:34.193540 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:34.193624 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:34.193760 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:34.193962 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.195277 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.196347 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.196957 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.197077 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.197651 (+ 574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:34.197654 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:34.197655 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:34.197849 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:34.197888 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:34.197902 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:34.197967 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:34.198020 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 14:59:34.198096 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":77,"threads_started":1}
I20260926 14:59:34.201287 24868 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:34.201913 24868 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:34.208514 24868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1: Generated new TSK 0
I20260926 14:59:34.208662 24868 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:34.213538 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:45023
--builtin_ntp_servers=127.7.37.212:42907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 14:59:34.300439 24881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:34.300614 24881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:34.300642 24881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:34.301702 24881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:34.301743 24881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:34.301775 24881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:59:34.303031 24881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:45023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:34.303236 24881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:34.303483 24881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:34.307808 24886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:34.307812 24887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:34.308082 24881 server_base.cc:1034] running on GCE node
W20260926 14:59:34.308190 24889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:34.308372 24881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:34.308594 24881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:34.309736 24881 hybrid_clock.cc:648] HybridClock initialized: now 1790434774309719 us; error 30 us; skew 500 ppm
I20260926 14:59:34.310858 24881 webserver.cc:460] Webserver started at http://127.7.37.193:45127/ using document root <none> and password file <none>
I20260926 14:59:34.311058 24881 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:34.311101 24881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:34.311216 24881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:34.312186 24881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "4bb6d1e2c38047debe854a2fcfd2d985"
format_stamp: "Formatted at 2026-09-26 14:59:34 on dist-test-slave-f9xb"
server_key: "9f9089810fdb4a5cbe76b03571bd3c07"
server_key_iv: "bb1e58922ee26b1de7ab01ddf4b0dd8b"
server_key_version: "encryptionkey@0"
I20260926 14:59:34.312485 24881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "4bb6d1e2c38047debe854a2fcfd2d985"
format_stamp: "Formatted at 2026-09-26 14:59:34 on dist-test-slave-f9xb"
server_key: "9f9089810fdb4a5cbe76b03571bd3c07"
server_key_iv: "bb1e58922ee26b1de7ab01ddf4b0dd8b"
server_key_version: "encryptionkey@0"
I20260926 14:59:34.313599 24881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:34.314260 24895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:34.314410 24881 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:34.314482 24881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "4bb6d1e2c38047debe854a2fcfd2d985"
format_stamp: "Formatted at 2026-09-26 14:59:34 on dist-test-slave-f9xb"
server_key: "9f9089810fdb4a5cbe76b03571bd3c07"
server_key_iv: "bb1e58922ee26b1de7ab01ddf4b0dd8b"
server_key_version: "encryptionkey@0"
I20260926 14:59:34.314543 24881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:34.325629 24881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:34.325848 24881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:34.325958 24881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:34.326128 24881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:34.326414 24881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:59:34.326447 24881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:34.326471 24881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:59:34.326488 24881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:34.331669 24881 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:33391
I20260926 14:59:34.331719 25008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:33391 every 8 connection(s)
I20260926 14:59:34.332027 24881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:59:34.336231 24864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.332606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:57847 (local address 127.7.37.254:45023)
0926 14:59:34.332661 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:34.332662 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:34.332844 (+ 182us) server_negotiation.cc:407] Connection header received
0926 14:59:34.333228 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:34.333229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:34.333237 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:34.333284 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:34.333821 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.334290 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.335221 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.335404 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.335455 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.336057 (+ 602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:34.336058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:34.336058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:34.336062 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:34.336074 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:34.336078 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:34.336115 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:34.336158 (+ 43us) server_negotiation.cc:299] Negotiation successful
0926 14:59:34.336179 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 14:59:34.336403 25011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.332590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:45023 (local address 127.7.37.193:57847)
0926 14:59:34.332747 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:34.332782 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:34.333165 (+ 383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:34.333328 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:34.333331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:34.333338 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:34.333764 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.333767 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.334344 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.334344 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.335154 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.335156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.335460 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.335461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.335555 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.335954 (+ 399us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:59:34.335955 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:34.335977 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:59:34.335996 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:34.336115 (+ 119us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:34.336117 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:34.336119 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:59:34.336159 (+ 40us) client_negotiation.cc:240] Negotiation successful
0926 14:59:34.336225 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":67,"threads_started":1}
I20260926 14:59:34.336810 25009 heartbeater.cc:347] Connected to a master server at 127.7.37.254:45023
I20260926 14:59:34.336884 25009 heartbeater.cc:464] Registering TS with master...
I20260926 14:59:34.337039 25009 heartbeater.cc:511] Master 127.7.37.254:45023 requested a full tablet report, sending...
I20260926 14:59:34.337450 24821 ts_manager.cc:194] Registered new tserver with Master: 4bb6d1e2c38047debe854a2fcfd2d985 (127.7.37.193:33391)
I20260926 14:59:34.337968 24821 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.37.193:57847
I20260926 14:59:34.338047 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 24881
I20260926 14:59:34.338136 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:59:34.338294 7319 external_mini_cluster.cc:1428] Setting key b5baa3ab25f16076945c9a1f5b97162d
I20260926 14:59:34.339037 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:45023
--builtin_ntp_servers=127.7.37.212:42907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 14:59:34.412901 25012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:34.413065 25012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:34.413085 25012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:34.414155 25012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:34.414191 25012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:34.414214 25012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:59:34.415472 25012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:45023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:34.415664 25012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:34.415935 25012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 14:59:34.420352 25012 server_base.cc:1034] running on GCE node
W20260926 14:59:34.420338 25018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:34.420326 25020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:34.420341 25017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:34.420708 25012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:34.420909 25012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:34.422039 25012 hybrid_clock.cc:648] HybridClock initialized: now 1790434774422028 us; error 27 us; skew 500 ppm
I20260926 14:59:34.423051 25012 webserver.cc:460] Webserver started at http://127.7.37.194:35587/ using document root <none> and password file <none>
I20260926 14:59:34.423254 25012 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:34.423297 25012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:34.423411 25012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:34.424337 25012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "24f312a44e29439ebf61c897ec3eacdf"
format_stamp: "Formatted at 2026-09-26 14:59:34 on dist-test-slave-f9xb"
server_key: "ae984e5a32695c018fd65d82a507b288"
server_key_iv: "43f606ff41a7dc9f0fa7c5b1bd0a6b1a"
server_key_version: "encryptionkey@0"
I20260926 14:59:34.424624 25012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "24f312a44e29439ebf61c897ec3eacdf"
format_stamp: "Formatted at 2026-09-26 14:59:34 on dist-test-slave-f9xb"
server_key: "ae984e5a32695c018fd65d82a507b288"
server_key_iv: "43f606ff41a7dc9f0fa7c5b1bd0a6b1a"
server_key_version: "encryptionkey@0"
I20260926 14:59:34.425704 25012 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:34.426283 25026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:34.426417 25012 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:34.426497 25012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "24f312a44e29439ebf61c897ec3eacdf"
format_stamp: "Formatted at 2026-09-26 14:59:34 on dist-test-slave-f9xb"
server_key: "ae984e5a32695c018fd65d82a507b288"
server_key_iv: "43f606ff41a7dc9f0fa7c5b1bd0a6b1a"
server_key_version: "encryptionkey@0"
I20260926 14:59:34.426553 25012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:34.449855 25012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:34.450060 25012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:34.450166 25012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:34.450338 25012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:34.450592 25012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:59:34.450623 25012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:34.450647 25012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:59:34.450670 25012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:34.455588 25012 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:35453
I20260926 14:59:34.455623 25139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:35453 every 8 connection(s)
I20260926 14:59:34.455892 25012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:59:34.460256 24864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.456433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:41489 (local address 127.7.37.254:45023)
0926 14:59:34.456488 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:34.456490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:34.456687 (+ 197us) server_negotiation.cc:407] Connection header received
0926 14:59:34.457100 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:34.457101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:34.457113 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:34.457160 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:34.457739 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.458208 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.459094 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.459297 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.459348 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.460014 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:34.460015 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:34.460015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:34.460020 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:34.460035 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:34.460038 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:34.460081 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:34.460156 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 14:59:34.460182 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 14:59:34.460446 25142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.456427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:45023 (local address 127.7.37.194:41489)
0926 14:59:34.456591 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:34.456618 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:34.457012 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:34.457217 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:34.457221 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:34.457238 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:34.457652 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.457656 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.458280 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.458281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.459018 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.459020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.459357 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.459358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.459477 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.459902 (+ 425us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:59:34.459902 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:34.459925 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:59:34.459944 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:34.460088 (+ 144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:34.460091 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:34.460094 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:59:34.460153 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 14:59:34.460269 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
I20260926 14:59:34.460798 25140 heartbeater.cc:347] Connected to a master server at 127.7.37.254:45023
I20260926 14:59:34.460877 25140 heartbeater.cc:464] Registering TS with master...
I20260926 14:59:34.461022 25140 heartbeater.cc:511] Master 127.7.37.254:45023 requested a full tablet report, sending...
I20260926 14:59:34.461331 24821 ts_manager.cc:194] Registered new tserver with Master: 24f312a44e29439ebf61c897ec3eacdf (127.7.37.194:35453)
I20260926 14:59:34.461668 24821 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.37.194:41489
I20260926 14:59:34.462793 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 25012
I20260926 14:59:34.462901 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:59:34.463091 7319 external_mini_cluster.cc:1428] Setting key 84b264701843762ba5fc77a88f2d98a2
I20260926 14:59:34.463874 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:45023
--builtin_ntp_servers=127.7.37.212:42907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 14:59:34.539029 25143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:34.539202 25143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:34.539222 25143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:34.540323 25143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:34.540359 25143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:34.540382 25143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:59:34.541607 25143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:42907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:45023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:34.541777 25143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:34.541985 25143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:34.546353 25149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:34.546444 25143 server_base.cc:1034] running on GCE node
W20260926 14:59:34.546360 25151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:34.546360 25148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:34.546706 25143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:34.546926 25143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:34.548048 25143 hybrid_clock.cc:648] HybridClock initialized: now 1790434774548027 us; error 40 us; skew 500 ppm
I20260926 14:59:34.549113 25143 webserver.cc:460] Webserver started at http://127.7.37.195:35645/ using document root <none> and password file <none>
I20260926 14:59:34.549310 25143 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:34.549355 25143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:34.549470 25143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:34.550395 25143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "2357bdcb17744a698e583a8a76d332dc"
format_stamp: "Formatted at 2026-09-26 14:59:34 on dist-test-slave-f9xb"
server_key: "e7f466d1c7fcf0715a6b3e5c47c62c83"
server_key_iv: "b6f250d1d9768ff32b1928fe983cd1df"
server_key_version: "encryptionkey@0"
I20260926 14:59:34.550688 25143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "2357bdcb17744a698e583a8a76d332dc"
format_stamp: "Formatted at 2026-09-26 14:59:34 on dist-test-slave-f9xb"
server_key: "e7f466d1c7fcf0715a6b3e5c47c62c83"
server_key_iv: "b6f250d1d9768ff32b1928fe983cd1df"
server_key_version: "encryptionkey@0"
I20260926 14:59:34.551797 25143 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:34.552424 25157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:34.552561 25143 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:34.552632 25143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "2357bdcb17744a698e583a8a76d332dc"
format_stamp: "Formatted at 2026-09-26 14:59:34 on dist-test-slave-f9xb"
server_key: "e7f466d1c7fcf0715a6b3e5c47c62c83"
server_key_iv: "b6f250d1d9768ff32b1928fe983cd1df"
server_key_version: "encryptionkey@0"
I20260926 14:59:34.552692 25143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:34.570401 25143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:34.570609 25143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:34.570713 25143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:34.570880 25143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:34.571131 25143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:59:34.571161 25143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:34.571185 25143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:59:34.571208 25143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:34.576237 25143 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:46569
I20260926 14:59:34.576272 25270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:46569 every 8 connection(s)
I20260926 14:59:34.576522 25143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:59:34.577793 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 25143
I20260926 14:59:34.577876 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:59:34.578022 7319 external_mini_cluster.cc:1428] Setting key cdde4cfbedd6da5b704114766dec06a9
I20260926 14:59:34.581310 24864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.577107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:44729 (local address 127.7.37.254:45023)
0926 14:59:34.577182 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:34.577184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:34.577328 (+ 144us) server_negotiation.cc:407] Connection header received
0926 14:59:34.578150 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:34.578151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:34.578162 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:34.578216 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:34.578952 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.579439 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.580336 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.580523 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.580572 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.581130 (+ 558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:34.581131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:34.581131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:34.581134 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:34.581147 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:34.581150 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:34.581184 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:34.581233 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 14:59:34.581254 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 14:59:34.581465 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.577069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:45023 (local address 127.7.37.195:44729)
0926 14:59:34.577224 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:34.577258 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:34.577631 (+ 373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:34.578294 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:34.578298 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:34.578307 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:34.578887 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.578891 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.579496 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.579496 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.580276 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.580278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.580578 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.580578 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.580668 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.581029 (+ 361us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:59:34.581030 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:34.581053 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:59:34.581071 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:34.581186 (+ 115us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:34.581188 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:34.581191 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:59:34.581230 (+ 39us) client_negotiation.cc:240] Negotiation successful
0926 14:59:34.581294 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":63,"threads_started":1}
I20260926 14:59:34.581863 25271 heartbeater.cc:347] Connected to a master server at 127.7.37.254:45023
I20260926 14:59:34.581938 25271 heartbeater.cc:464] Registering TS with master...
I20260926 14:59:34.582062 25271 heartbeater.cc:511] Master 127.7.37.254:45023 requested a full tablet report, sending...
I20260926 14:59:34.582355 24821 ts_manager.cc:194] Registered new tserver with Master: 2357bdcb17744a698e583a8a76d332dc (127.7.37.195:46569)
I20260926 14:59:34.582664 24821 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.37.195:44729
I20260926 14:59:34.589462 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 14:59:34.593293 24864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.590619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38124 (local address 127.7.37.254:45023)
0926 14:59:34.590669 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:34.590670 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:34.590758 (+ 88us) server_negotiation.cc:407] Connection header received
0926 14:59:34.590811 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:34.590812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:34.590823 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:34.590866 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:34.591014 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.591457 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.592262 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.592441 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.592491 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.593048 (+ 557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:34.593050 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:34.593050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:34.593054 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:34.593065 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:34.593068 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:34.593135 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:34.593188 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 14:59:34.593206 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 14:59:34.594460 24821 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38124:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:59:34.596271 24821 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:59:34.601588 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.597932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:33391 (local address 127.0.0.1:48880)
0926 14:59:34.598193 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:34.598239 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:34.598299 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:34.598544 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:34.598546 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:34.598548 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:34.598615 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.598617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.599896 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.599897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.600613 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.600615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.601319 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.601320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.601390 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.601402 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 14:59:34.601474 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 14:59:34.601522 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":45,"threads_started":1}
I20260926 14:59:34.601610 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.598133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48880 (local address 127.7.37.193:33391)
0926 14:59:34.598278 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:34.598282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:34.598292 (+ 10us) server_negotiation.cc:407] Connection header received
0926 14:59:34.598380 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:34.598382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:34.598394 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:34.598466 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:34.598671 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.599807 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.600690 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.601238 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.601307 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.601383 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:34.601488 (+ 105us) server_negotiation.cc:299] Negotiation successful
0926 14:59:34.601523 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260926 14:59:34.601953 25279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.597933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:46569 (local address 127.0.0.1:59420)
0926 14:59:34.598374 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:34.598383 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:34.598435 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:34.598595 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:34.598595 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:34.598598 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:34.598639 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.598641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.599545 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.599546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.600952 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.600954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.601755 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.601755 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.601812 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.601815 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:59:34.601856 (+ 41us) client_negotiation.cc:240] Negotiation successful
0926 14:59:34.601891 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":85,"threads_started":1}
I20260926 14:59:34.602020 25282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.598014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34196 (local address 127.7.37.194:35453)
0926 14:59:34.598327 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:34.598331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:34.598504 (+ 173us) server_negotiation.cc:407] Connection header received
0926 14:59:34.598541 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:34.598543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:34.598558 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:34.598631 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:34.598802 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.599868 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.601025 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.601715 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.601781 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.601862 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:34.601907 (+ 45us) server_negotiation.cc:299] Negotiation successful
0926 14:59:34.601942 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":83,"threads_started":1}
I20260926 14:59:34.602038 25280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.597933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:35453 (local address 127.0.0.1:34196)
0926 14:59:34.598408 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:34.598415 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:34.598455 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:34.598699 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:34.598700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:34.598702 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:34.598729 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.598730 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.599940 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.599941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.600957 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.600958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.601782 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.601783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.601841 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.601845 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:59:34.601894 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 14:59:34.601931 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"spinlock_wait_cycles":1408,"thread_start_us":86,"threads_started":1}
I20260926 14:59:34.602284 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.598105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59420 (local address 127.7.37.195:46569)
0926 14:59:34.598226 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:34.598230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:34.598437 (+ 207us) server_negotiation.cc:407] Connection header received
0926 14:59:34.598501 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:34.598503 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:34.598516 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:34.598583 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:34.598707 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.599476 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.601028 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.601586 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.601642 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.601723 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:34.602169 (+ 446us) server_negotiation.cc:299] Negotiation successful
0926 14:59:34.602203 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":59,"threads_started":1}
I20260926 14:59:34.602587 24943 tablet_service.cc:1467] Processing CreateTablet for tablet b88960d68ecb41c19920e0616cbb0dfc (DEFAULT_TABLE table=test-table [id=17826824460d48d1b84297348844a13d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:59:34.602846 24943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b88960d68ecb41c19920e0616cbb0dfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:34.602778 25205 tablet_service.cc:1467] Processing CreateTablet for tablet b88960d68ecb41c19920e0616cbb0dfc (DEFAULT_TABLE table=test-table [id=17826824460d48d1b84297348844a13d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:59:34.603009 25205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b88960d68ecb41c19920e0616cbb0dfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:34.605553 25286 tablet_bootstrap.cc:492] T b88960d68ecb41c19920e0616cbb0dfc P 4bb6d1e2c38047debe854a2fcfd2d985: Bootstrap starting.
I20260926 14:59:34.605619 25285 tablet_bootstrap.cc:492] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc: Bootstrap starting.
I20260926 14:59:34.606009 25074 tablet_service.cc:1467] Processing CreateTablet for tablet b88960d68ecb41c19920e0616cbb0dfc (DEFAULT_TABLE table=test-table [id=17826824460d48d1b84297348844a13d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:59:34.606187 25286 tablet_bootstrap.cc:654] T b88960d68ecb41c19920e0616cbb0dfc P 4bb6d1e2c38047debe854a2fcfd2d985: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:34.606220 25285 tablet_bootstrap.cc:654] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:34.606297 25074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b88960d68ecb41c19920e0616cbb0dfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:34.606392 25286 log.cc:824] T b88960d68ecb41c19920e0616cbb0dfc P 4bb6d1e2c38047debe854a2fcfd2d985: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:34.606482 25285 log.cc:824] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:34.606948 25286 tablet_bootstrap.cc:492] T b88960d68ecb41c19920e0616cbb0dfc P 4bb6d1e2c38047debe854a2fcfd2d985: No bootstrap required, opened a new log
I20260926 14:59:34.606988 25286 ts_tablet_manager.cc:1397] T b88960d68ecb41c19920e0616cbb0dfc P 4bb6d1e2c38047debe854a2fcfd2d985: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:59:34.608242 25285 tablet_bootstrap.cc:492] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc: No bootstrap required, opened a new log
I20260926 14:59:34.608296 25285 ts_tablet_manager.cc:1397] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 14:59:34.608731 25289 tablet_bootstrap.cc:492] T b88960d68ecb41c19920e0616cbb0dfc P 24f312a44e29439ebf61c897ec3eacdf: Bootstrap starting.
I20260926 14:59:34.608697 25286 raft_consensus.cc:348] T b88960d68ecb41c19920e0616cbb0dfc P 4bb6d1e2c38047debe854a2fcfd2d985 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f312a44e29439ebf61c897ec3eacdf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 35453 } } peers { permanent_uuid: "4bb6d1e2c38047debe854a2fcfd2d985" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33391 } } peers { permanent_uuid: "2357bdcb17744a698e583a8a76d332dc" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 46569 } }
I20260926 14:59:34.608846 25286 raft_consensus.cc:374] T b88960d68ecb41c19920e0616cbb0dfc P 4bb6d1e2c38047debe854a2fcfd2d985 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:34.608876 25286 raft_consensus.cc:729] T b88960d68ecb41c19920e0616cbb0dfc P 4bb6d1e2c38047debe854a2fcfd2d985 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bb6d1e2c38047debe854a2fcfd2d985, State: Initialized, Role: FOLLOWER
I20260926 14:59:34.608985 25286 consensus_queue.cc:260] T b88960d68ecb41c19920e0616cbb0dfc P 4bb6d1e2c38047debe854a2fcfd2d985 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f312a44e29439ebf61c897ec3eacdf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 35453 } } peers { permanent_uuid: "4bb6d1e2c38047debe854a2fcfd2d985" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33391 } } peers { permanent_uuid: "2357bdcb17744a698e583a8a76d332dc" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 46569 } }
I20260926 14:59:34.609210 25286 ts_tablet_manager.cc:1428] T b88960d68ecb41c19920e0616cbb0dfc P 4bb6d1e2c38047debe854a2fcfd2d985: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:59:34.609336 25009 heartbeater.cc:503] Master 127.7.37.254:45023 was elected leader, sending a full tablet report...
I20260926 14:59:34.609535 25289 tablet_bootstrap.cc:654] T b88960d68ecb41c19920e0616cbb0dfc P 24f312a44e29439ebf61c897ec3eacdf: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:34.609762 25289 log.cc:824] T b88960d68ecb41c19920e0616cbb0dfc P 24f312a44e29439ebf61c897ec3eacdf: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:34.610329 25289 tablet_bootstrap.cc:492] T b88960d68ecb41c19920e0616cbb0dfc P 24f312a44e29439ebf61c897ec3eacdf: No bootstrap required, opened a new log
I20260926 14:59:34.610381 25289 ts_tablet_manager.cc:1397] T b88960d68ecb41c19920e0616cbb0dfc P 24f312a44e29439ebf61c897ec3eacdf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:59:34.610450 25285 raft_consensus.cc:348] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f312a44e29439ebf61c897ec3eacdf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 35453 } } peers { permanent_uuid: "4bb6d1e2c38047debe854a2fcfd2d985" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33391 } } peers { permanent_uuid: "2357bdcb17744a698e583a8a76d332dc" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 46569 } }
I20260926 14:59:34.610571 25285 raft_consensus.cc:374] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:34.610610 25285 raft_consensus.cc:729] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2357bdcb17744a698e583a8a76d332dc, State: Initialized, Role: FOLLOWER
I20260926 14:59:34.610713 25285 consensus_queue.cc:260] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f312a44e29439ebf61c897ec3eacdf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 35453 } } peers { permanent_uuid: "4bb6d1e2c38047debe854a2fcfd2d985" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33391 } } peers { permanent_uuid: "2357bdcb17744a698e583a8a76d332dc" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 46569 } }
I20260926 14:59:34.610917 25285 ts_tablet_manager.cc:1428] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 14:59:34.610934 25271 heartbeater.cc:503] Master 127.7.37.254:45023 was elected leader, sending a full tablet report...
I20260926 14:59:34.612340 25289 raft_consensus.cc:348] T b88960d68ecb41c19920e0616cbb0dfc P 24f312a44e29439ebf61c897ec3eacdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f312a44e29439ebf61c897ec3eacdf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 35453 } } peers { permanent_uuid: "4bb6d1e2c38047debe854a2fcfd2d985" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33391 } } peers { permanent_uuid: "2357bdcb17744a698e583a8a76d332dc" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 46569 } }
I20260926 14:59:34.612450 25289 raft_consensus.cc:374] T b88960d68ecb41c19920e0616cbb0dfc P 24f312a44e29439ebf61c897ec3eacdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:34.612479 25289 raft_consensus.cc:729] T b88960d68ecb41c19920e0616cbb0dfc P 24f312a44e29439ebf61c897ec3eacdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f312a44e29439ebf61c897ec3eacdf, State: Initialized, Role: FOLLOWER
I20260926 14:59:34.612576 25289 consensus_queue.cc:260] T b88960d68ecb41c19920e0616cbb0dfc P 24f312a44e29439ebf61c897ec3eacdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f312a44e29439ebf61c897ec3eacdf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 35453 } } peers { permanent_uuid: "4bb6d1e2c38047debe854a2fcfd2d985" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33391 } } peers { permanent_uuid: "2357bdcb17744a698e583a8a76d332dc" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 46569 } }
I20260926 14:59:34.612767 25289 ts_tablet_manager.cc:1428] T b88960d68ecb41c19920e0616cbb0dfc P 24f312a44e29439ebf61c897ec3eacdf: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 14:59:34.612811 25140 heartbeater.cc:503] Master 127.7.37.254:45023 was elected leader, sending a full tablet report...
W20260926 14:59:34.706338 25141 tablet.cc:2369] T b88960d68ecb41c19920e0616cbb0dfc P 24f312a44e29439ebf61c897ec3eacdf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:59:34.711218 25292 raft_consensus.cc:482] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:59:34.711299 25292 raft_consensus.cc:504] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f312a44e29439ebf61c897ec3eacdf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 35453 } } peers { permanent_uuid: "4bb6d1e2c38047debe854a2fcfd2d985" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33391 } } peers { permanent_uuid: "2357bdcb17744a698e583a8a76d332dc" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 46569 } }
I20260926 14:59:34.711599 25292 leader_election.cc:288] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24f312a44e29439ebf61c897ec3eacdf (127.7.37.194:35453), 4bb6d1e2c38047debe854a2fcfd2d985 (127.7.37.193:33391)
I20260926 14:59:34.714123 25282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.711800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:45993 (local address 127.7.37.194:35453)
0926 14:59:34.711841 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:34.711844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:34.711853 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:59:34.711900 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:34.711901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:34.711909 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:34.711970 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:34.712104 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.712600 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.713492 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.713835 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.713886 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.713893 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:34.714025 (+ 132us) server_negotiation.cc:299] Negotiation successful
0926 14:59:34.714063 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 14:59:34.714149 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.711813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:49109 (local address 127.7.37.193:33391)
0926 14:59:34.711863 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:34.711865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:34.711968 (+ 103us) server_negotiation.cc:407] Connection header received
0926 14:59:34.712013 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:34.712014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:34.712026 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:34.712077 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:34.712204 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.712626 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.713517 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.713862 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.713924 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.713931 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:34.714058 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 14:59:34.714095 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 14:59:34.714159 25294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.711738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:33391 (local address 127.7.37.195:49109)
0926 14:59:34.711910 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:34.711919 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:34.711968 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:34.712083 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:34.712084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:34.712085 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:34.712148 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.712150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.712695 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.712696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.713439 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.713441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.713932 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.713933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.713995 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.713999 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:59:34.714047 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 14:59:34.714081 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":79,"threads_started":1}
I20260926 14:59:34.714141 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.711730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:35453 (local address 127.7.37.195:45993)
0926 14:59:34.711792 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:34.711803 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:34.711842 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:34.711979 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:34.711980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:34.711981 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:34.712047 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.712049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.712669 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.712671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.713417 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:34.713419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.713905 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.713906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:34.713971 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.713973 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:59:34.714011 (+ 38us) client_negotiation.cc:240] Negotiation successful
0926 14:59:34.714042 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":26}
I20260926 14:59:34.714404 24963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b88960d68ecb41c19920e0616cbb0dfc" candidate_uuid: "2357bdcb17744a698e583a8a76d332dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bb6d1e2c38047debe854a2fcfd2d985" is_pre_election: true
I20260926 14:59:34.714524 24963 raft_consensus.cc:2454] T b88960d68ecb41c19920e0616cbb0dfc P 4bb6d1e2c38047debe854a2fcfd2d985 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2357bdcb17744a698e583a8a76d332dc in term 0.
I20260926 14:59:34.714509 25094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b88960d68ecb41c19920e0616cbb0dfc" candidate_uuid: "2357bdcb17744a698e583a8a76d332dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f312a44e29439ebf61c897ec3eacdf" is_pre_election: true
I20260926 14:59:34.714624 25094 raft_consensus.cc:2454] T b88960d68ecb41c19920e0616cbb0dfc P 24f312a44e29439ebf61c897ec3eacdf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2357bdcb17744a698e583a8a76d332dc in term 0.
I20260926 14:59:34.714715 25158 leader_election.cc:302] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2357bdcb17744a698e583a8a76d332dc, 4bb6d1e2c38047debe854a2fcfd2d985; no voters:
I20260926 14:59:34.714852 25292 raft_consensus.cc:2781] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:59:34.714905 25292 raft_consensus.cc:482] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:59:34.714927 25292 raft_consensus.cc:3037] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:34.715521 25292 raft_consensus.cc:504] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f312a44e29439ebf61c897ec3eacdf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 35453 } } peers { permanent_uuid: "4bb6d1e2c38047debe854a2fcfd2d985" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33391 } } peers { permanent_uuid: "2357bdcb17744a698e583a8a76d332dc" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 46569 } }
I20260926 14:59:34.715628 25292 leader_election.cc:288] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [CANDIDATE]: Term 1 election: Requested vote from peers 24f312a44e29439ebf61c897ec3eacdf (127.7.37.194:35453), 4bb6d1e2c38047debe854a2fcfd2d985 (127.7.37.193:33391)
I20260926 14:59:34.715793 25094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b88960d68ecb41c19920e0616cbb0dfc" candidate_uuid: "2357bdcb17744a698e583a8a76d332dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f312a44e29439ebf61c897ec3eacdf"
I20260926 14:59:34.715860 25094 raft_consensus.cc:3037] T b88960d68ecb41c19920e0616cbb0dfc P 24f312a44e29439ebf61c897ec3eacdf [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:34.716466 25094 raft_consensus.cc:2454] T b88960d68ecb41c19920e0616cbb0dfc P 24f312a44e29439ebf61c897ec3eacdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2357bdcb17744a698e583a8a76d332dc in term 1.
I20260926 14:59:34.716558 24963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b88960d68ecb41c19920e0616cbb0dfc" candidate_uuid: "2357bdcb17744a698e583a8a76d332dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bb6d1e2c38047debe854a2fcfd2d985"
I20260926 14:59:34.716639 25161 leader_election.cc:302] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2357bdcb17744a698e583a8a76d332dc, 24f312a44e29439ebf61c897ec3eacdf; no voters:
I20260926 14:59:34.716632 24963 raft_consensus.cc:3037] T b88960d68ecb41c19920e0616cbb0dfc P 4bb6d1e2c38047debe854a2fcfd2d985 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:34.716760 25292 raft_consensus.cc:2781] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:34.716882 25292 raft_consensus.cc:686] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [term 1 LEADER]: Becoming Leader. State: Replica: 2357bdcb17744a698e583a8a76d332dc, State: Running, Role: LEADER
I20260926 14:59:34.716959 25292 consensus_queue.cc:237] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f312a44e29439ebf61c897ec3eacdf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 35453 } } peers { permanent_uuid: "4bb6d1e2c38047debe854a2fcfd2d985" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33391 } } peers { permanent_uuid: "2357bdcb17744a698e583a8a76d332dc" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 46569 } }
I20260926 14:59:34.717504 24963 raft_consensus.cc:2454] T b88960d68ecb41c19920e0616cbb0dfc P 4bb6d1e2c38047debe854a2fcfd2d985 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2357bdcb17744a698e583a8a76d332dc in term 1.
I20260926 14:59:34.717725 24821 catalog_manager.cc:5495] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 2357bdcb17744a698e583a8a76d332dc (127.7.37.195). New cstate: current_term: 1 leader_uuid: "2357bdcb17744a698e583a8a76d332dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f312a44e29439ebf61c897ec3eacdf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 35453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bb6d1e2c38047debe854a2fcfd2d985" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2357bdcb17744a698e583a8a76d332dc" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 46569 } health_report { overall_health: HEALTHY } } }
I20260926 14:59:34.743070 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.741215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59436 (local address 127.7.37.195:46569)
0926 14:59:34.741262 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:34.741264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:34.741271 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:59:34.741299 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:34.741299 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:34.741311 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:34.741362 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:59:34.741448 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.741859 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.742499 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:34.742659 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:34.742704 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:34.742805 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:59:34.742884 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:59:34.742935 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:34.742963 (+ 28us) server_negotiation.cc:299] Negotiation successful
0926 14:59:34.742976 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 14:59:34.744472 25094 raft_consensus.cc:1261] T b88960d68ecb41c19920e0616cbb0dfc P 24f312a44e29439ebf61c897ec3eacdf [term 1 FOLLOWER]: Refusing update from remote peer 2357bdcb17744a698e583a8a76d332dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:59:34.744482 24963 raft_consensus.cc:1261] T b88960d68ecb41c19920e0616cbb0dfc P 4bb6d1e2c38047debe854a2fcfd2d985 [term 1 FOLLOWER]: Refusing update from remote peer 2357bdcb17744a698e583a8a76d332dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:59:34.744766 25295 consensus_queue.cc:1035] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f312a44e29439ebf61c897ec3eacdf" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 35453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:59:34.744863 25292 consensus_queue.cc:1035] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bb6d1e2c38047debe854a2fcfd2d985" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:59:34.748503 24821 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:59:34.748581 24821 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:59:34.749219 24821 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9c2d1cfd7a9e43b7a89479fc2ab571c1: Sending DeleteTablet for 3 replicas of tablet b88960d68ecb41c19920e0616cbb0dfc
I20260926 14:59:34.749470 25205 tablet_service.cc:1514] Processing DeleteTablet for tablet b88960d68ecb41c19920e0616cbb0dfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:59:34 UTC) from {username='slave'} at 127.0.0.1:59420
I20260926 14:59:34.749526 24943 tablet_service.cc:1514] Processing DeleteTablet for tablet b88960d68ecb41c19920e0616cbb0dfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:59:34 UTC) from {username='slave'} at 127.0.0.1:48880
I20260926 14:59:34.749526 24942 tablet_service.cc:1514] Processing DeleteTablet for tablet b88960d68ecb41c19920e0616cbb0dfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:59:34 UTC) from {username='slave'} at 127.0.0.1:48880
I20260926 14:59:34.749698 25308 tablet_replica.cc:330] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc: stopping tablet replica
I20260926 14:59:34.749787 25308 raft_consensus.cc:2229] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:59:34.749783 25074 tablet_service.cc:1514] Processing DeleteTablet for tablet b88960d68ecb41c19920e0616cbb0dfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:59:34 UTC) from {username='slave'} at 127.0.0.1:34196
I20260926 14:59:34.749882 25308 raft_consensus.cc:2258] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:59:34.750213 25309 tablet_replica.cc:330] T b88960d68ecb41c19920e0616cbb0dfc P 4bb6d1e2c38047debe854a2fcfd2d985: stopping tablet replica
I20260926 14:59:34.750264 25308 ts_tablet_manager.cc:1905] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:59:34.750274 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 24881
I20260926 14:59:34.750322 25309 raft_consensus.cc:2229] T b88960d68ecb41c19920e0616cbb0dfc P 4bb6d1e2c38047debe854a2fcfd2d985 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:59:34.750429 25309 raft_consensus.cc:2258] T b88960d68ecb41c19920e0616cbb0dfc P 4bb6d1e2c38047debe854a2fcfd2d985 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:59:34.751994 25308 ts_tablet_manager.cc:1918] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:59:34.752051 25308 log.cc:1196] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/b88960d68ecb41c19920e0616cbb0dfc
I20260926 14:59:34.752277 25308 ts_tablet_manager.cc:1939] T b88960d68ecb41c19920e0616cbb0dfc P 2357bdcb17744a698e583a8a76d332dc: Deleting consensus metadata
I20260926 14:59:34.752591 24807 catalog_manager.cc:4841] TS 2357bdcb17744a698e583a8a76d332dc (127.7.37.195:46569): tablet b88960d68ecb41c19920e0616cbb0dfc (table test-table [id=17826824460d48d1b84297348844a13d]) successfully deleted
W20260926 14:59:34.756615 24805 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.37.193:33391 (error 108)
I20260926 14:59:34.756851 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 25012
I20260926 14:59:34.760031 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.759932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:33391 (local address 127.0.0.1:48896)
0926 14:59:34.759973 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:34.760007 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.193:33391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260926 14:59:34.760181 24805 catalog_manager.cc:4584] TS 4bb6d1e2c38047debe854a2fcfd2d985 (127.7.37.193:33391): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b88960d68ecb41c19920e0616cbb0dfc: Network error: Client connection negotiation failed: client connection to 127.7.37.193:33391: connect: Connection refused (error 111)
I20260926 14:59:34.761566 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:34.761481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:35453 (local address 127.0.0.1:34208)
0926 14:59:34.761517 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:34.761540 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.194:35453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260926 14:59:34.761576 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 25143
I20260926 14:59:34.765815 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 24790
2026-09-26T14:59:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (721 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:34 dist-test-slave-f9xb krb5kdc[25314](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:34 dist-test-slave-f9xb krb5kdc[25314](info): set up 2 sockets
Sep 26 14:59:34 dist-test-slave-f9xb krb5kdc[25314](info): commencing operation
krb5kdc: starting...
W20260926 14:59:35.707088 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:35 dist-test-slave-f9xb krb5kdc[25314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:59:35.810523 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36871
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:38527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:36871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:35.885629 25327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:35.885799 25327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:35.885828 25327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:35.886894 25327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:35.886935 25327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:35.886955 25327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:35.886969 25327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:35.886986 25327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:35.888371 25327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:38527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:36871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:35.888579 25327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:35.888768 25327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:35.892953 25333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:35.892971 25332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:35.893033 25335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:35.893270 25327 server_base.cc:1034] running on GCE node
I20260926 14:59:35.893455 25327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:35.893661 25327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:35.894809 25327 hybrid_clock.cc:648] HybridClock initialized: now 1790434775894798 us; error 33 us; skew 500 ppm
Sep 26 14:59:35 dist-test-slave-f9xb krb5kdc[25314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434775, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:35.897022 25327 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:59:35.897378 25327 webserver.cc:460] Webserver started at http://127.7.37.254:41963/ using document root <none> and password file <none>
I20260926 14:59:35.897564 25327 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:35.897614 25327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:35.897728 25327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:35.898636 25327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "c9d12bc97ae94c839fae5eda03b29422"
format_stamp: "Formatted at 2026-09-26 14:59:35 on dist-test-slave-f9xb"
server_key: "ff283ff6fc6c1356208b4d0625e0425b"
server_key_iv: "ed232454be43e25b0028839bb66be602"
server_key_version: "encryptionkey@0"
I20260926 14:59:35.898931 25327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "c9d12bc97ae94c839fae5eda03b29422"
format_stamp: "Formatted at 2026-09-26 14:59:35 on dist-test-slave-f9xb"
server_key: "ff283ff6fc6c1356208b4d0625e0425b"
server_key_iv: "ed232454be43e25b0028839bb66be602"
server_key_version: "encryptionkey@0"
I20260926 14:59:35.900090 25327 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260926 14:59:35.900676 25342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:35.900808 25327 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:35.900874 25327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "c9d12bc97ae94c839fae5eda03b29422"
format_stamp: "Formatted at 2026-09-26 14:59:35 on dist-test-slave-f9xb"
server_key: "ff283ff6fc6c1356208b4d0625e0425b"
server_key_iv: "ed232454be43e25b0028839bb66be602"
server_key_version: "encryptionkey@0"
I20260926 14:59:35.900933 25327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:35.937415 25327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:35.939021 25327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:35.939141 25327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:35.942538 25327 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:36871
I20260926 14:59:35.942569 25394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:36871 every 8 connection(s)
I20260926 14:59:35.942836 25327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:35.943540 25395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:35.944253 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 25327
I20260926 14:59:35.944366 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:35.944547 7319 external_mini_cluster.cc:1428] Setting key d50215dcd646397c0aa1672c0fca6871
I20260926 14:59:35.946322 25395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422: Bootstrap starting.
I20260926 14:59:35.946846 25395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:35.947082 25395 log.cc:824] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:35.947631 25395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422: No bootstrap required, opened a new log
I20260926 14:59:35.949391 25395 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d12bc97ae94c839fae5eda03b29422" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36871 } }
I20260926 14:59:35.949509 25395 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:35.949529 25395 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9d12bc97ae94c839fae5eda03b29422, State: Initialized, Role: FOLLOWER
I20260926 14:59:35.949625 25395 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d12bc97ae94c839fae5eda03b29422" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36871 } }
I20260926 14:59:35.949689 25395 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:35.949719 25395 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:35.949761 25395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:35.950349 25395 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d12bc97ae94c839fae5eda03b29422" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36871 } }
I20260926 14:59:35.950451 25395 leader_election.cc:302] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9d12bc97ae94c839fae5eda03b29422; no voters:
Sep 26 14:59:35 dist-test-slave-f9xb krb5kdc[25314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:59:35.950600 25395 leader_election.cc:288] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:35.950628 25400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:35.950724 25400 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422 [term 1 LEADER]: Becoming Leader. State: Replica: c9d12bc97ae94c839fae5eda03b29422, State: Running, Role: LEADER
I20260926 14:59:35.950839 25395 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:35.950851 25400 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d12bc97ae94c839fae5eda03b29422" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36871 } }
I20260926 14:59:35.951212 25402 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9d12bc97ae94c839fae5eda03b29422. Latest consensus state: current_term: 1 leader_uuid: "c9d12bc97ae94c839fae5eda03b29422" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d12bc97ae94c839fae5eda03b29422" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36871 } } }
I20260926 14:59:35.951292 25402 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:35.951436 25401 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9d12bc97ae94c839fae5eda03b29422" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d12bc97ae94c839fae5eda03b29422" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36871 } } }
I20260926 14:59:35.951522 25401 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:35.951978 25404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:35.953176 25404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:35.954012 25398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:35.944998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42528 (local address 127.7.37.254:36871)
0926 14:59:35.945141 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:35.945149 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:35.945171 (+ 22us) server_negotiation.cc:407] Connection header received
0926 14:59:35.945530 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:35.945536 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:35.945575 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:35.945674 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:35.946405 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:35.947300 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:35.948126 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:35.948542 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:35.948631 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:35.951005 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:35.951010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:35.951010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:35.951021 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:35.952732 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:35.953050 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:35.953054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:35.953054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:35.953133 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:35.953309 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:35.953310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:35.953311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:35.953518 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:35.953651 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:35.953745 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 14:59:35.953867 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260926 14:59:35.954707 25404 catalog_manager.cc:1347] Generated new cluster ID: 8325b36ae59b4cd3b195ae6e139b07dd
I20260926 14:59:35.954756 25404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:59:35.958688 25404 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:35.959331 25404 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:35.962327 25404 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422: Generated new TSK 0
I20260926 14:59:35.962461 25404 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
I20260926 14:59:35.992305 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36871
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:38527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:36.097288 25421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:36.097463 25421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:36.097484 25421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:36.098574 25421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:36.098610 25421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:36.098634 25421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:59:36.100045 25421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:38527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:36.100250 25421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:36.100464 25421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:36.104838 25426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:36.104838 25427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:36.104980 25429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:36.105131 25421 server_base.cc:1034] running on GCE node
I20260926 14:59:36.105350 25421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:36.105573 25421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:36.106711 25421 hybrid_clock.cc:648] HybridClock initialized: now 1790434776106702 us; error 31 us; skew 500 ppm
Sep 26 14:59:36 dist-test-slave-f9xb krb5kdc[25314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434776, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:36.108897 25421 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:59:36.109270 25421 webserver.cc:460] Webserver started at http://127.7.37.193:35967/ using document root <none> and password file <none>
I20260926 14:59:36.109455 25421 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:36.109500 25421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:36.109614 25421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:36.110549 25421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "3ab8247edb314398828be27f76ba7ec6"
format_stamp: "Formatted at 2026-09-26 14:59:36 on dist-test-slave-f9xb"
server_key: "0cbb2030a07fafa04391f1559d08a4dd"
server_key_iv: "db7caecc67f6c1b896eaeb3db5db236a"
server_key_version: "encryptionkey@0"
I20260926 14:59:36.110862 25421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "3ab8247edb314398828be27f76ba7ec6"
format_stamp: "Formatted at 2026-09-26 14:59:36 on dist-test-slave-f9xb"
server_key: "0cbb2030a07fafa04391f1559d08a4dd"
server_key_iv: "db7caecc67f6c1b896eaeb3db5db236a"
server_key_version: "encryptionkey@0"
I20260926 14:59:36.112007 25421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 14:59:36.112597 25436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:36.112771 25421 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:59:36.112846 25421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "3ab8247edb314398828be27f76ba7ec6"
format_stamp: "Formatted at 2026-09-26 14:59:36 on dist-test-slave-f9xb"
server_key: "0cbb2030a07fafa04391f1559d08a4dd"
server_key_iv: "db7caecc67f6c1b896eaeb3db5db236a"
server_key_version: "encryptionkey@0"
I20260926 14:59:36.112901 25421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:36.135797 25421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:36.137420 25421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:36.137542 25421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:36.137732 25421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:36.137986 25421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:59:36.138018 25421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:36.138044 25421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:59:36.138059 25421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:36.143291 25421 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:40877
I20260926 14:59:36.143327 25549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:40877 every 8 connection(s)
I20260926 14:59:36.143590 25421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:59:36.146555 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 25421
I20260926 14:59:36.146648 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:59:36.146824 7319 external_mini_cluster.cc:1428] Setting key 26910a1a8a55858a69bbdb7fb7228ef7
Sep 26 14:59:36 dist-test-slave-f9xb krb5kdc[25314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434776, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:59:36.152182 25398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.144187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:42527 (local address 127.7.37.254:36871)
0926 14:59:36.144241 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:36.144243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:36.144448 (+ 205us) server_negotiation.cc:407] Connection header received
0926 14:59:36.144880 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:36.144881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:36.144890 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:36.144936 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:36.145741 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.146224 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.147193 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.147484 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.147549 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:36.149469 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:36.149470 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:36.149470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:36.149477 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:36.150720 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:36.151190 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:36.151190 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:36.151191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:36.151241 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:36.151534 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:36.151535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:36.151535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:36.151640 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:36.151691 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:36.152003 (+ 312us) server_negotiation.cc:299] Negotiation successful
0926 14:59:36.152071 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 14:59:36.152362 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.144185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36871 (local address 127.7.37.193:42527)
0926 14:59:36.144347 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:36.144379 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:36.144812 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:36.144986 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:36.144989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:36.145487 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:36.145667 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:36.145671 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.146286 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.146287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:36.147130 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:36.147132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.147564 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.147565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:36.147690 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:36.148131 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:59:36.148138 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:36.149391 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:36.150792 (+ 1401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:36.150798 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:36.150800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:36.151120 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:36.151296 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:36.151296 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:36.151297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:36.151423 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:36.151700 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:36.151707 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:36.151880 (+ 173us) client_negotiation.cc:769] Sending connection context
0926 14:59:36.151993 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 14:59:36.152159 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260926 14:59:36.152804 25550 heartbeater.cc:347] Connected to a master server at 127.7.37.254:36871
I20260926 14:59:36.152886 25550 heartbeater.cc:464] Registering TS with master...
I20260926 14:59:36.153034 25550 heartbeater.cc:511] Master 127.7.37.254:36871 requested a full tablet report, sending...
I20260926 14:59:36.153518 25359 ts_manager.cc:194] Registered new tserver with Master: 3ab8247edb314398828be27f76ba7ec6 (127.7.37.193:40877)
WARNING: no policy specified for kudu/127.7.37.194@KRBTEST.COM; defaulting to no policy
I20260926 14:59:36.154381 25359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:42527
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
I20260926 14:59:36.167832 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36871
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:38527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:36.244314 25555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:36.244483 25555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:36.244503 25555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:36.245581 25555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:36.245627 25555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:36.245653 25555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:59:36.247035 25555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:38527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:36.247228 25555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:36.247417 25555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 14:59:36.251899 25555 server_base.cc:1034] running on GCE node
W20260926 14:59:36.252005 25560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:36.251899 25561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:36.251936 25563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:36.252359 25555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:36.252576 25555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:36.253720 25555 hybrid_clock.cc:648] HybridClock initialized: now 1790434776253704 us; error 31 us; skew 500 ppm
Sep 26 14:59:36 dist-test-slave-f9xb krb5kdc[25314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434776, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:36.256038 25555 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:59:36.256424 25555 webserver.cc:460] Webserver started at http://127.7.37.194:46521/ using document root <none> and password file <none>
I20260926 14:59:36.256615 25555 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:36.256659 25555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:36.256771 25555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:36.257653 25555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "29ab2e5ba27840a4a38f172ce9bb67ae"
format_stamp: "Formatted at 2026-09-26 14:59:36 on dist-test-slave-f9xb"
server_key: "28d5120c79712d0c4e36a18f2b683396"
server_key_iv: "f99110a6cd24caacf38a956deb83c7af"
server_key_version: "encryptionkey@0"
I20260926 14:59:36.257953 25555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "29ab2e5ba27840a4a38f172ce9bb67ae"
format_stamp: "Formatted at 2026-09-26 14:59:36 on dist-test-slave-f9xb"
server_key: "28d5120c79712d0c4e36a18f2b683396"
server_key_iv: "f99110a6cd24caacf38a956deb83c7af"
server_key_version: "encryptionkey@0"
I20260926 14:59:36.259085 25555 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 14:59:36.259768 25570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:36.259945 25555 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:36.260020 25555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "29ab2e5ba27840a4a38f172ce9bb67ae"
format_stamp: "Formatted at 2026-09-26 14:59:36 on dist-test-slave-f9xb"
server_key: "28d5120c79712d0c4e36a18f2b683396"
server_key_iv: "f99110a6cd24caacf38a956deb83c7af"
server_key_version: "encryptionkey@0"
I20260926 14:59:36.260082 25555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:36.268610 25555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:36.270375 25555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:36.270484 25555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:36.270651 25555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:36.270903 25555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:59:36.270937 25555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:36.270961 25555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:59:36.270984 25555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:36.276393 25555 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:43865
I20260926 14:59:36.276443 25683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:43865 every 8 connection(s)
I20260926 14:59:36.276693 25555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:59:36.281327 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 25555
I20260926 14:59:36.281421 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:59:36.281597 7319 external_mini_cluster.cc:1428] Setting key 02ff3826535b0726641c8ba5014219bc
Sep 26 14:59:36 dist-test-slave-f9xb krb5kdc[25314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434776, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:59:36.286352 25398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.277373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:41707 (local address 127.7.37.254:36871)
0926 14:59:36.277422 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:36.277424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:36.277605 (+ 181us) server_negotiation.cc:407] Connection header received
0926 14:59:36.277994 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:36.277995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:36.278010 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:36.278102 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:36.278926 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.279712 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.280628 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.280923 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.280996 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:36.283288 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:36.283290 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:36.283291 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:36.283299 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:36.284933 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:36.285281 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:36.285281 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:36.285282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:36.285341 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:36.285650 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:36.285651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:36.285651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:36.285783 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:36.285826 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:36.286171 (+ 345us) server_negotiation.cc:299] Negotiation successful
0926 14:59:36.286260 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 14:59:36.287056 25686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.277341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36871 (local address 127.7.37.194:41707)
0926 14:59:36.277481 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:36.277516 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:36.277914 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:36.278142 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:36.278145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:36.278683 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:36.278848 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:36.278852 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.279831 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.279832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:36.280558 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:36.280560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.280991 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.280992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:36.281075 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:36.281519 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:59:36.281529 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:36.283173 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:36.285012 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:36.285018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:36.285019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:36.285217 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:36.285427 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:36.285428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:36.285429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:36.285558 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:36.285874 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:36.285880 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:36.286024 (+ 144us) client_negotiation.cc:769] Sending connection context
0926 14:59:36.286685 (+ 661us) client_negotiation.cc:240] Negotiation successful
0926 14:59:36.286850 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
I20260926 14:59:36.287534 25684 heartbeater.cc:347] Connected to a master server at 127.7.37.254:36871
I20260926 14:59:36.287674 25684 heartbeater.cc:464] Registering TS with master...
I20260926 14:59:36.287919 25684 heartbeater.cc:511] Master 127.7.37.254:36871 requested a full tablet report, sending...
I20260926 14:59:36.288349 25359 ts_manager.cc:194] Registered new tserver with Master: 29ab2e5ba27840a4a38f172ce9bb67ae (127.7.37.194:43865)
I20260926 14:59:36.288837 25359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:41707
WARNING: no policy specified for kudu/127.7.37.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
I20260926 14:59:36.304714 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36871
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:38527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:36.379415 25689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:36.379581 25689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:36.379601 25689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:36.380693 25689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:36.380733 25689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:36.380759 25689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:59:36.382107 25689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:38527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:36.382280 25689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:36.382424 25689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:36.386823 25694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:36.386852 25689 server_base.cc:1034] running on GCE node
W20260926 14:59:36.386823 25695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:36.386861 25697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:36.387247 25689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:36.387449 25689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:36.388574 25689 hybrid_clock.cc:648] HybridClock initialized: now 1790434776388563 us; error 26 us; skew 500 ppm
Sep 26 14:59:36 dist-test-slave-f9xb krb5kdc[25314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434776, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:36.390715 25689 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:59:36.391108 25689 webserver.cc:460] Webserver started at http://127.7.37.195:42687/ using document root <none> and password file <none>
I20260926 14:59:36.391300 25689 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:36.391343 25689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:36.391458 25689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:36.392373 25689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "358113132995495496e423db5fa16c27"
format_stamp: "Formatted at 2026-09-26 14:59:36 on dist-test-slave-f9xb"
server_key: "4bef91c37feb19b5d6c7ba16b5740b3d"
server_key_iv: "8c65430f479b6fd917ff03dc46a3afc6"
server_key_version: "encryptionkey@0"
I20260926 14:59:36.392676 25689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "358113132995495496e423db5fa16c27"
format_stamp: "Formatted at 2026-09-26 14:59:36 on dist-test-slave-f9xb"
server_key: "4bef91c37feb19b5d6c7ba16b5740b3d"
server_key_iv: "8c65430f479b6fd917ff03dc46a3afc6"
server_key_version: "encryptionkey@0"
I20260926 14:59:36.393769 25689 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 14:59:36.394374 25704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:36.394512 25689 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:59:36.394585 25689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "358113132995495496e423db5fa16c27"
format_stamp: "Formatted at 2026-09-26 14:59:36 on dist-test-slave-f9xb"
server_key: "4bef91c37feb19b5d6c7ba16b5740b3d"
server_key_iv: "8c65430f479b6fd917ff03dc46a3afc6"
server_key_version: "encryptionkey@0"
I20260926 14:59:36.394651 25689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:36.409965 25689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:36.411379 25689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:36.411497 25689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:36.411666 25689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:36.411957 25689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:59:36.411988 25689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:36.412014 25689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:59:36.412030 25689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:36.417129 25689 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:44021
I20260926 14:59:36.417160 25817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:44021 every 8 connection(s)
I20260926 14:59:36.417398 25689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:59:36.418156 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 25689
I20260926 14:59:36.418267 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:59:36.418440 7319 external_mini_cluster.cc:1428] Setting key 61c5bbe955c1339ffced903c9f5e2117
Sep 26 14:59:36 dist-test-slave-f9xb krb5kdc[25314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434776, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:59:36.425182 25398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.417938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:49731 (local address 127.7.37.254:36871)
0926 14:59:36.417988 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:36.417990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:36.418155 (+ 165us) server_negotiation.cc:407] Connection header received
0926 14:59:36.418607 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:36.418609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:36.418621 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:36.418689 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:36.419563 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.420098 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.420948 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.421126 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.421171 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:36.422998 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:36.423000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:36.423000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:36.423006 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:36.424063 (+ 1057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:36.424397 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:36.424398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:36.424398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:36.424445 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:36.424643 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:36.424644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:36.424644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:36.424747 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:36.424799 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:36.425028 (+ 229us) server_negotiation.cc:299] Negotiation successful
0926 14:59:36.425098 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 14:59:36.425328 25820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.417920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36871 (local address 127.7.37.195:49731)
0926 14:59:36.418052 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:36.418087 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:36.418523 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:36.418751 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:36.418755 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:36.419332 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:36.419488 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:36.419492 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.420159 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.420160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:36.420888 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:36.420890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.421197 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.421198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:36.421286 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:36.421696 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:59:36.421704 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:36.422918 (+ 1214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:36.424128 (+ 1210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:36.424133 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:36.424135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:36.424335 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:36.424502 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:36.424503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:36.424503 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:36.424591 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:36.424798 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:36.424804 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:36.424920 (+ 116us) client_negotiation.cc:769] Sending connection context
0926 14:59:36.424998 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 14:59:36.425134 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":64,"threads_started":1}
I20260926 14:59:36.425685 25818 heartbeater.cc:347] Connected to a master server at 127.7.37.254:36871
I20260926 14:59:36.425755 25818 heartbeater.cc:464] Registering TS with master...
I20260926 14:59:36.425868 25818 heartbeater.cc:511] Master 127.7.37.254:36871 requested a full tablet report, sending...
I20260926 14:59:36.426144 25359 ts_manager.cc:194] Registered new tserver with Master: 358113132995495496e423db5fa16c27 (127.7.37.195:44021)
I20260926 14:59:36.426479 25359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:49731
I20260926 14:59:36.429890 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 14:59:36.436180 25398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.431010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42530 (local address 127.7.37.254:36871)
0926 14:59:36.431047 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:36.431049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:36.431144 (+ 95us) server_negotiation.cc:407] Connection header received
0926 14:59:36.431226 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:36.431227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:36.431235 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:36.431278 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:36.431777 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.432213 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.432901 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.433074 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.433123 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:36.434198 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:36.434199 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:36.434199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:36.434205 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:36.435121 (+ 916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:36.435447 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:36.435448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:36.435448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:36.435493 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:36.435667 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:36.435668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:36.435668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:36.435844 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:36.435937 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:36.436045 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 14:59:36.436100 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 14:59:36.437392 25359 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:59:36.438838 25359 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:59:36.444387 25826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.440628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:43865 (local address 127.0.0.1:57334)
0926 14:59:36.440777 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:36.440810 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:36.440871 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:36.441091 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:36.441093 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:36.441095 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:36.441190 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:36.441192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.442222 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.442223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:36.443274 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:36.443276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.444118 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.444119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:36.444192 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:36.444204 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 14:59:36.444276 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 14:59:36.444323 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":61,"threads_started":1}
I20260926 14:59:36.444415 25829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.440766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57334 (local address 127.7.37.194:43865)
0926 14:59:36.440950 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:36.440954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:36.440963 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:59:36.440994 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:36.440996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:36.441006 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:36.441074 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:36.441276 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.442138 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.443358 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.444033 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.444104 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:36.444167 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:36.444295 (+ 128us) server_negotiation.cc:299] Negotiation successful
0926 14:59:36.444334 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":84,"threads_started":1}
I20260926 14:59:36.444712 25830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.440639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:40877 (local address 127.0.0.1:37686)
0926 14:59:36.440916 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:36.440926 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:36.440972 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:36.441340 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:36.441340 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:36.441342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:36.441373 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:36.441374 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.442650 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.442651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:36.443698 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:36.443700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.444495 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.444496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:36.444553 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:36.444557 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:59:36.444608 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 14:59:36.444651 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":28,"thread_start_us":81,"threads_started":1}
I20260926 14:59:36.444757 25828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.440750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37686 (local address 127.7.37.193:40877)
0926 14:59:36.441131 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:36.441136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:36.441145 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:59:36.441181 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:36.441183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:36.441198 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:36.441263 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:36.441436 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.442548 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.443782 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.444349 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.444406 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:36.444461 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:36.444639 (+ 178us) server_negotiation.cc:299] Negotiation successful
0926 14:59:36.444686 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":63,"threads_started":1}
I20260926 14:59:36.444779 25831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.440952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35034 (local address 127.7.37.195:44021)
0926 14:59:36.441089 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:36.441092 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:36.441217 (+ 125us) server_negotiation.cc:407] Connection header received
0926 14:59:36.441273 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:36.441274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:36.441287 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:36.441351 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:36.441477 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.442570 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.443548 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.444272 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.444330 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:36.444376 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:36.444687 (+ 311us) server_negotiation.cc:299] Negotiation successful
0926 14:59:36.444721 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260926 14:59:36.444631 25827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.440816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:44021 (local address 127.0.0.1:35034)
0926 14:59:36.441160 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:36.441169 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:36.441213 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:36.441361 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:36.441361 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:36.441364 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:36.441407 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:36.441408 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.442673 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.442674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:36.443465 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:36.443467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.444410 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.444411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:36.444469 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:36.444472 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:59:36.444519 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 14:59:36.444556 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":23,"thread_start_us":64,"threads_started":1}
I20260926 14:59:36.445353 25618 tablet_service.cc:1467] Processing CreateTablet for tablet 62ab902a1c2b42f4853d58926a5bf728 (DEFAULT_TABLE table=test-table [id=a6d38289a9804256b215649548ab0430]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:59:36.445449 25752 tablet_service.cc:1467] Processing CreateTablet for tablet 62ab902a1c2b42f4853d58926a5bf728 (DEFAULT_TABLE table=test-table [id=a6d38289a9804256b215649548ab0430]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:59:36.445542 25484 tablet_service.cc:1467] Processing CreateTablet for tablet 62ab902a1c2b42f4853d58926a5bf728 (DEFAULT_TABLE table=test-table [id=a6d38289a9804256b215649548ab0430]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:59:36.445669 25618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62ab902a1c2b42f4853d58926a5bf728. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:36.445685 25752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62ab902a1c2b42f4853d58926a5bf728. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:36.445816 25484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62ab902a1c2b42f4853d58926a5bf728. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:36.447983 25832 tablet_bootstrap.cc:492] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27: Bootstrap starting.
I20260926 14:59:36.448555 25832 tablet_bootstrap.cc:654] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:36.448850 25834 tablet_bootstrap.cc:492] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae: Bootstrap starting.
I20260926 14:59:36.449030 25833 tablet_bootstrap.cc:492] T 62ab902a1c2b42f4853d58926a5bf728 P 3ab8247edb314398828be27f76ba7ec6: Bootstrap starting.
I20260926 14:59:36.449262 25832 log.cc:824] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:36.449565 25833 tablet_bootstrap.cc:654] T 62ab902a1c2b42f4853d58926a5bf728 P 3ab8247edb314398828be27f76ba7ec6: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:36.449874 25832 tablet_bootstrap.cc:492] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27: No bootstrap required, opened a new log
I20260926 14:59:36.449928 25832 ts_tablet_manager.cc:1397] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:59:36.450007 25833 log.cc:824] T 62ab902a1c2b42f4853d58926a5bf728 P 3ab8247edb314398828be27f76ba7ec6: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:36.450692 25834 tablet_bootstrap.cc:654] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:36.451129 25834 log.cc:824] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:36.451354 25832 raft_consensus.cc:348] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab8247edb314398828be27f76ba7ec6" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40877 } } peers { permanent_uuid: "29ab2e5ba27840a4a38f172ce9bb67ae" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43865 } } peers { permanent_uuid: "358113132995495496e423db5fa16c27" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 44021 } }
I20260926 14:59:36.451512 25832 raft_consensus.cc:374] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:36.451550 25832 raft_consensus.cc:729] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 358113132995495496e423db5fa16c27, State: Initialized, Role: FOLLOWER
I20260926 14:59:36.451644 25833 tablet_bootstrap.cc:492] T 62ab902a1c2b42f4853d58926a5bf728 P 3ab8247edb314398828be27f76ba7ec6: No bootstrap required, opened a new log
I20260926 14:59:36.451710 25833 ts_tablet_manager.cc:1397] T 62ab902a1c2b42f4853d58926a5bf728 P 3ab8247edb314398828be27f76ba7ec6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 14:59:36.451699 25832 consensus_queue.cc:260] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab8247edb314398828be27f76ba7ec6" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40877 } } peers { permanent_uuid: "29ab2e5ba27840a4a38f172ce9bb67ae" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43865 } } peers { permanent_uuid: "358113132995495496e423db5fa16c27" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 44021 } }
I20260926 14:59:36.451972 25832 ts_tablet_manager.cc:1428] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:59:36.452031 25834 tablet_bootstrap.cc:492] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae: No bootstrap required, opened a new log
I20260926 14:59:36.452064 25818 heartbeater.cc:503] Master 127.7.37.254:36871 was elected leader, sending a full tablet report...
I20260926 14:59:36.452135 25834 ts_tablet_manager.cc:1397] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 14:59:36.453593 25833 raft_consensus.cc:348] T 62ab902a1c2b42f4853d58926a5bf728 P 3ab8247edb314398828be27f76ba7ec6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab8247edb314398828be27f76ba7ec6" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40877 } } peers { permanent_uuid: "29ab2e5ba27840a4a38f172ce9bb67ae" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43865 } } peers { permanent_uuid: "358113132995495496e423db5fa16c27" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 44021 } }
I20260926 14:59:36.453763 25833 raft_consensus.cc:374] T 62ab902a1c2b42f4853d58926a5bf728 P 3ab8247edb314398828be27f76ba7ec6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:36.453802 25833 raft_consensus.cc:729] T 62ab902a1c2b42f4853d58926a5bf728 P 3ab8247edb314398828be27f76ba7ec6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab8247edb314398828be27f76ba7ec6, State: Initialized, Role: FOLLOWER
I20260926 14:59:36.453904 25833 consensus_queue.cc:260] T 62ab902a1c2b42f4853d58926a5bf728 P 3ab8247edb314398828be27f76ba7ec6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab8247edb314398828be27f76ba7ec6" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40877 } } peers { permanent_uuid: "29ab2e5ba27840a4a38f172ce9bb67ae" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43865 } } peers { permanent_uuid: "358113132995495496e423db5fa16c27" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 44021 } }
I20260926 14:59:36.453960 25834 raft_consensus.cc:348] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab8247edb314398828be27f76ba7ec6" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40877 } } peers { permanent_uuid: "29ab2e5ba27840a4a38f172ce9bb67ae" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43865 } } peers { permanent_uuid: "358113132995495496e423db5fa16c27" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 44021 } }
I20260926 14:59:36.454119 25834 raft_consensus.cc:374] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:36.454171 25550 heartbeater.cc:503] Master 127.7.37.254:36871 was elected leader, sending a full tablet report...
I20260926 14:59:36.454197 25834 raft_consensus.cc:729] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29ab2e5ba27840a4a38f172ce9bb67ae, State: Initialized, Role: FOLLOWER
I20260926 14:59:36.454180 25833 ts_tablet_manager.cc:1428] T 62ab902a1c2b42f4853d58926a5bf728 P 3ab8247edb314398828be27f76ba7ec6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 14:59:36.454334 25834 consensus_queue.cc:260] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab8247edb314398828be27f76ba7ec6" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40877 } } peers { permanent_uuid: "29ab2e5ba27840a4a38f172ce9bb67ae" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43865 } } peers { permanent_uuid: "358113132995495496e423db5fa16c27" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 44021 } }
I20260926 14:59:36.454553 25834 ts_tablet_manager.cc:1428] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:59:36.454669 25684 heartbeater.cc:503] Master 127.7.37.254:36871 was elected leader, sending a full tablet report...
W20260926 14:59:36.527164 25685 tablet.cc:2369] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 14:59:36.644225 25551 tablet.cc:2369] T 62ab902a1c2b42f4853d58926a5bf728 P 3ab8247edb314398828be27f76ba7ec6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 14:59:36.667869 25819 tablet.cc:2369] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 14:59:36.708479 25838 raft_consensus.cc:482] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:59:36.708585 25838 raft_consensus.cc:504] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab8247edb314398828be27f76ba7ec6" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40877 } } peers { permanent_uuid: "29ab2e5ba27840a4a38f172ce9bb67ae" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43865 } } peers { permanent_uuid: "358113132995495496e423db5fa16c27" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 44021 } }
I20260926 14:59:36.708917 25838 leader_election.cc:288] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ab8247edb314398828be27f76ba7ec6 (127.7.37.193:40877), 29ab2e5ba27840a4a38f172ce9bb67ae (127.7.37.194:43865)
I20260926 14:59:36.711547 25828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.709095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:38045 (local address 127.7.37.193:40877)
0926 14:59:36.709145 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:36.709147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:36.709156 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:59:36.709212 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:36.709213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:36.709226 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:36.709275 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:36.709425 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.709941 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.710893 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.711247 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.711298 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:36.711306 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:36.711448 (+ 142us) server_negotiation.cc:299] Negotiation successful
0926 14:59:36.711484 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 14:59:36.711547 25829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.709189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:54057 (local address 127.7.37.194:43865)
0926 14:59:36.709233 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:36.709235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:36.709320 (+ 85us) server_negotiation.cc:407] Connection header received
0926 14:59:36.709352 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:36.709353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:36.709361 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:36.709415 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:36.709540 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.710006 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.710899 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.711263 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.711326 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:36.711334 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:36.711456 (+ 122us) server_negotiation.cc:299] Negotiation successful
0926 14:59:36.711487 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:59:36.711571 25820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.709061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:40877 (local address 127.7.37.195:38045)
0926 14:59:36.709101 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:36.709111 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:36.709151 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:36.709284 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:36.709284 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:36.709286 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:36.709354 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:36.709356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.710018 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.710019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:36.710806 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:36.710808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.711312 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.711313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:36.711386 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:36.711389 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:59:36.711431 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 14:59:36.711469 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":34}
I20260926 14:59:36.711573 25841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.709065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:43865 (local address 127.7.37.195:54057)
0926 14:59:36.709262 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:36.709270 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:36.709313 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:36.709424 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:36.709425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:36.709426 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:36.709487 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:36.709488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.710075 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.710076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:36.710815 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:36.710817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.711338 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.711339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:36.711393 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:36.711396 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:59:36.711442 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 14:59:36.711483 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":40,"thread_start_us":95,"threads_started":1}
I20260926 14:59:36.711891 25504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ab902a1c2b42f4853d58926a5bf728" candidate_uuid: "358113132995495496e423db5fa16c27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab8247edb314398828be27f76ba7ec6" is_pre_election: true
I20260926 14:59:36.711983 25638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ab902a1c2b42f4853d58926a5bf728" candidate_uuid: "358113132995495496e423db5fa16c27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29ab2e5ba27840a4a38f172ce9bb67ae" is_pre_election: true
I20260926 14:59:36.712026 25504 raft_consensus.cc:2454] T 62ab902a1c2b42f4853d58926a5bf728 P 3ab8247edb314398828be27f76ba7ec6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 358113132995495496e423db5fa16c27 in term 0.
I20260926 14:59:36.712118 25638 raft_consensus.cc:2454] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 358113132995495496e423db5fa16c27 in term 0.
I20260926 14:59:36.712244 25708 leader_election.cc:302] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 358113132995495496e423db5fa16c27, 3ab8247edb314398828be27f76ba7ec6; no voters:
I20260926 14:59:36.712375 25838 raft_consensus.cc:2781] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:59:36.712426 25838 raft_consensus.cc:482] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:59:36.712445 25838 raft_consensus.cc:3037] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:36.713042 25838 raft_consensus.cc:504] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab8247edb314398828be27f76ba7ec6" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40877 } } peers { permanent_uuid: "29ab2e5ba27840a4a38f172ce9bb67ae" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43865 } } peers { permanent_uuid: "358113132995495496e423db5fa16c27" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 44021 } }
I20260926 14:59:36.713148 25838 leader_election.cc:288] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [CANDIDATE]: Term 1 election: Requested vote from peers 3ab8247edb314398828be27f76ba7ec6 (127.7.37.193:40877), 29ab2e5ba27840a4a38f172ce9bb67ae (127.7.37.194:43865)
I20260926 14:59:36.713289 25638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ab902a1c2b42f4853d58926a5bf728" candidate_uuid: "358113132995495496e423db5fa16c27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29ab2e5ba27840a4a38f172ce9bb67ae"
I20260926 14:59:36.713304 25504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ab902a1c2b42f4853d58926a5bf728" candidate_uuid: "358113132995495496e423db5fa16c27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab8247edb314398828be27f76ba7ec6"
I20260926 14:59:36.713351 25638 raft_consensus.cc:3037] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:36.713372 25504 raft_consensus.cc:3037] T 62ab902a1c2b42f4853d58926a5bf728 P 3ab8247edb314398828be27f76ba7ec6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:36.713966 25638 raft_consensus.cc:2454] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 358113132995495496e423db5fa16c27 in term 1.
I20260926 14:59:36.714051 25504 raft_consensus.cc:2454] T 62ab902a1c2b42f4853d58926a5bf728 P 3ab8247edb314398828be27f76ba7ec6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 358113132995495496e423db5fa16c27 in term 1.
I20260926 14:59:36.714121 25705 leader_election.cc:302] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29ab2e5ba27840a4a38f172ce9bb67ae, 358113132995495496e423db5fa16c27; no voters:
I20260926 14:59:36.714232 25838 raft_consensus.cc:2781] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:36.714345 25838 raft_consensus.cc:686] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [term 1 LEADER]: Becoming Leader. State: Replica: 358113132995495496e423db5fa16c27, State: Running, Role: LEADER
I20260926 14:59:36.714498 25838 consensus_queue.cc:237] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab8247edb314398828be27f76ba7ec6" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40877 } } peers { permanent_uuid: "29ab2e5ba27840a4a38f172ce9bb67ae" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43865 } } peers { permanent_uuid: "358113132995495496e423db5fa16c27" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 44021 } }
I20260926 14:59:36.715207 25359 catalog_manager.cc:5495] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 reported cstate change: term changed from 0 to 1, leader changed from <none> to 358113132995495496e423db5fa16c27 (127.7.37.195). New cstate: current_term: 1 leader_uuid: "358113132995495496e423db5fa16c27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab8247edb314398828be27f76ba7ec6" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29ab2e5ba27840a4a38f172ce9bb67ae" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "358113132995495496e423db5fa16c27" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 44021 } health_report { overall_health: HEALTHY } } }
I20260926 14:59:36.723794 25831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.721545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35046 (local address 127.7.37.195:44021)
0926 14:59:36.721598 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:36.721600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:36.721608 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:59:36.721664 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:36.721665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:36.721680 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:36.721738 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:59:36.721844 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.722258 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.722928 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.723119 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.723274 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:36.723447 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:59:36.723532 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:59:36.723610 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:36.723692 (+ 82us) server_negotiation.cc:299] Negotiation successful
0926 14:59:36.723699 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 14:59:36.725232 25638 raft_consensus.cc:1261] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae [term 1 FOLLOWER]: Refusing update from remote peer 358113132995495496e423db5fa16c27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:59:36.725232 25504 raft_consensus.cc:1261] T 62ab902a1c2b42f4853d58926a5bf728 P 3ab8247edb314398828be27f76ba7ec6 [term 1 FOLLOWER]: Refusing update from remote peer 358113132995495496e423db5fa16c27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:59:36.725538 25838 consensus_queue.cc:1035] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29ab2e5ba27840a4a38f172ce9bb67ae" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:59:36.725663 25842 consensus_queue.cc:1035] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab8247edb314398828be27f76ba7ec6" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 40877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:59:36.728507 25847 mvcc.cc:205] Tried to move back new op lower bound from 7333620845464428544 to 7333620845423063040. Current Snapshot: MvccSnapshot[applied={T|T < 7333620845464428544}]
I20260926 14:59:36.728559 25846 mvcc.cc:205] Tried to move back new op lower bound from 7333620845464428544 to 7333620845423063040. Current Snapshot: MvccSnapshot[applied={T|T < 7333620845464428544}]
I20260926 14:59:36.729754 25359 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:59:36.729844 25359 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:59:36.730465 25359 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9d12bc97ae94c839fae5eda03b29422: Sending DeleteTablet for 3 replicas of tablet 62ab902a1c2b42f4853d58926a5bf728
I20260926 14:59:36.730806 25484 tablet_service.cc:1514] Processing DeleteTablet for tablet 62ab902a1c2b42f4853d58926a5bf728 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:59:36 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:37686
I20260926 14:59:36.730822 25618 tablet_service.cc:1514] Processing DeleteTablet for tablet 62ab902a1c2b42f4853d58926a5bf728 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:59:36 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:57334
I20260926 14:59:36.731011 25854 tablet_replica.cc:330] T 62ab902a1c2b42f4853d58926a5bf728 P 3ab8247edb314398828be27f76ba7ec6: stopping tablet replica
I20260926 14:59:36.730996 25752 tablet_service.cc:1514] Processing DeleteTablet for tablet 62ab902a1c2b42f4853d58926a5bf728 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:59:36 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:35034
I20260926 14:59:36.731101 25854 raft_consensus.cc:2229] T 62ab902a1c2b42f4853d58926a5bf728 P 3ab8247edb314398828be27f76ba7ec6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:59:36.731251 25854 raft_consensus.cc:2258] T 62ab902a1c2b42f4853d58926a5bf728 P 3ab8247edb314398828be27f76ba7ec6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:59:36.731262 25855 tablet_replica.cc:330] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae: stopping tablet replica
I20260926 14:59:36.731344 25855 raft_consensus.cc:2229] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:59:36.731428 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 25421
I20260926 14:59:36.731890 25855 raft_consensus.cc:2258] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:59:36.732311 25855 ts_tablet_manager.cc:1905] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:59:36.733917 25855 ts_tablet_manager.cc:1918] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:59:36.733974 25855 log.cc:1196] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/62ab902a1c2b42f4853d58926a5bf728
I20260926 14:59:36.734215 25855 ts_tablet_manager.cc:1939] T 62ab902a1c2b42f4853d58926a5bf728 P 29ab2e5ba27840a4a38f172ce9bb67ae: Deleting consensus metadata
I20260926 14:59:36.734577 25343 catalog_manager.cc:4841] TS 29ab2e5ba27840a4a38f172ce9bb67ae (127.7.37.194:43865): tablet 62ab902a1c2b42f4853d58926a5bf728 (table test-table [id=a6d38289a9804256b215649548ab0430]) successfully deleted
W20260926 14:59:36.736740 25346 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.37.193:40877 (error 108)
I20260926 14:59:36.731176 25856 tablet_replica.cc:330] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27: stopping tablet replica
I20260926 14:59:36.736857 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 25555
I20260926 14:59:36.736894 25856 raft_consensus.cc:2229] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:59:36.736965 25856 raft_consensus.cc:2258] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:59:36.737838 25826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.737092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:40877 (local address 127.0.0.1:37694)
0926 14:59:36.737757 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:36.737809 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.193:40877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":645}
W20260926 14:59:36.737985 25346 catalog_manager.cc:4584] TS 3ab8247edb314398828be27f76ba7ec6 (127.7.37.193:40877): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 62ab902a1c2b42f4853d58926a5bf728: Network error: Client connection negotiation failed: client connection to 127.7.37.193:40877: connect: Connection refused (error 111)
I20260926 14:59:36.738688 25856 ts_tablet_manager.cc:1905] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:59:36.740309 25856 ts_tablet_manager.cc:1918] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:59:36.740374 25856 log.cc:1196] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/62ab902a1c2b42f4853d58926a5bf728
I20260926 14:59:36.740659 25856 ts_tablet_manager.cc:1939] T 62ab902a1c2b42f4853d58926a5bf728 P 358113132995495496e423db5fa16c27: Deleting consensus metadata
I20260926 14:59:36.741222 25346 catalog_manager.cc:4841] TS 358113132995495496e423db5fa16c27 (127.7.37.195:44021): tablet 62ab902a1c2b42f4853d58926a5bf728 (table test-table [id=a6d38289a9804256b215649548ab0430]) successfully deleted
I20260926 14:59:36.742075 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 25689
I20260926 14:59:36.745868 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 25327
2026-09-26T14:59:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1980 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-26T14:59:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:36Z Disabled control of system clock
I20260926 14:59:36.769781 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:38527
--webserver_interface=127.7.37.254
--webserver_port=0
--builtin_ntp_servers=127.7.37.212:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:38527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 14:59:36.844080 25861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:36.844275 25861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:36.844296 25861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:36.845361 25861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:36.845399 25861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:36.845412 25861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:36.845425 25861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:36.845438 25861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:36.846711 25861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:38527
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:38527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:36.846904 25861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:36.847136 25861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:36.851372 25869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:36.851408 25861 server_base.cc:1034] running on GCE node
W20260926 14:59:36.851460 25867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:36.851471 25866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:36.851861 25861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:36.852108 25861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:36.853248 25861 hybrid_clock.cc:648] HybridClock initialized: now 1790434776853238 us; error 27 us; skew 500 ppm
I20260926 14:59:36.854328 25861 webserver.cc:460] Webserver started at http://127.7.37.254:44337/ using document root <none> and password file <none>
I20260926 14:59:36.854537 25861 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:36.854579 25861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:36.854697 25861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:36.855621 25861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "b31bf6cb47cc46c48e16f2710964e6a7"
format_stamp: "Formatted at 2026-09-26 14:59:36 on dist-test-slave-f9xb"
server_key: "dbbd495c49d4d1a34e0b72fcb972ef82"
server_key_iv: "3d1c9b78e50ad566fd5de07bf629c131"
server_key_version: "encryptionkey@0"
I20260926 14:59:36.855937 25861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "b31bf6cb47cc46c48e16f2710964e6a7"
format_stamp: "Formatted at 2026-09-26 14:59:36 on dist-test-slave-f9xb"
server_key: "dbbd495c49d4d1a34e0b72fcb972ef82"
server_key_iv: "3d1c9b78e50ad566fd5de07bf629c131"
server_key_version: "encryptionkey@0"
I20260926 14:59:36.857009 25861 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 14:59:36.857605 25875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:36.857738 25861 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:59:36.857810 25861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "b31bf6cb47cc46c48e16f2710964e6a7"
format_stamp: "Formatted at 2026-09-26 14:59:36 on dist-test-slave-f9xb"
server_key: "dbbd495c49d4d1a34e0b72fcb972ef82"
server_key_iv: "3d1c9b78e50ad566fd5de07bf629c131"
server_key_version: "encryptionkey@0"
I20260926 14:59:36.857872 25861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:36.878862 25861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:36.879079 25861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:36.879195 25861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:36.882479 25861 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:38527
I20260926 14:59:36.882507 25927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:38527 every 8 connection(s)
I20260926 14:59:36.882756 25861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:36.883152 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 25861
I20260926 14:59:36.883248 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:36.883464 7319 external_mini_cluster.cc:1428] Setting key f197637663fefb89642158d69358c5a8
I20260926 14:59:36.883724 25928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:36.886088 25928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7: Bootstrap starting.
I20260926 14:59:36.886574 25928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:36.886811 25928 log.cc:824] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:36.887482 25928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7: No bootstrap required, opened a new log
I20260926 14:59:36.888358 25931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:36.883907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33928 (local address 127.7.37.254:38527)
0926 14:59:36.884106 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:36.884113 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:36.884134 (+ 21us) server_negotiation.cc:407] Connection header received
0926 14:59:36.884650 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:36.884655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:36.884708 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:36.884816 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:36.885020 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.885829 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.886615 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:36.887089 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:36.887200 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:36.887810 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:36.887814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:36.887815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:36.887952 (+ 137us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:36.887978 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:36.887988 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:36.888035 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:36.888118 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 14:59:36.888170 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260926 14:59:36.889300 25928 raft_consensus.cc:348] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b31bf6cb47cc46c48e16f2710964e6a7" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 38527 } }
I20260926 14:59:36.889418 25928 raft_consensus.cc:374] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:36.889448 25928 raft_consensus.cc:729] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b31bf6cb47cc46c48e16f2710964e6a7, State: Initialized, Role: FOLLOWER
I20260926 14:59:36.889552 25928 consensus_queue.cc:260] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b31bf6cb47cc46c48e16f2710964e6a7" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 38527 } }
I20260926 14:59:36.889628 25928 raft_consensus.cc:388] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:36.889665 25928 raft_consensus.cc:482] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:36.889709 25928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:36.890170 25928 raft_consensus.cc:504] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b31bf6cb47cc46c48e16f2710964e6a7" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 38527 } }
I20260926 14:59:36.890257 25928 leader_election.cc:302] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b31bf6cb47cc46c48e16f2710964e6a7; no voters:
I20260926 14:59:36.890398 25928 leader_election.cc:288] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:36.890446 25933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:36.890568 25933 raft_consensus.cc:686] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7 [term 1 LEADER]: Becoming Leader. State: Replica: b31bf6cb47cc46c48e16f2710964e6a7, State: Running, Role: LEADER
I20260926 14:59:36.890625 25928 sys_catalog.cc:564] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:36.890673 25933 consensus_queue.cc:237] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b31bf6cb47cc46c48e16f2710964e6a7" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 38527 } }
I20260926 14:59:36.890937 25933 sys_catalog.cc:455] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b31bf6cb47cc46c48e16f2710964e6a7. Latest consensus state: current_term: 1 leader_uuid: "b31bf6cb47cc46c48e16f2710964e6a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b31bf6cb47cc46c48e16f2710964e6a7" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 38527 } } }
I20260926 14:59:36.890971 25934 sys_catalog.cc:455] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b31bf6cb47cc46c48e16f2710964e6a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b31bf6cb47cc46c48e16f2710964e6a7" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 38527 } } }
I20260926 14:59:36.891188 25934 sys_catalog.cc:458] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:36.891623 25933 sys_catalog.cc:458] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:36.891760 25942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:36.892262 25942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:36.893467 25942 catalog_manager.cc:1347] Generated new cluster ID: 17fb20b62cea4a749c2f44377d2c719a
I20260926 14:59:36.893507 25942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:59:36.917354 25942 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:36.917783 25942 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:36.922243 25942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7: Generated new TSK 0
I20260926 14:59:36.922366 25942 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:36.929714 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:38527
--builtin_ntp_servers=127.7.37.212:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 14:59:37.005497 25952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:37.005686 25952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:37.005714 25952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:37.006811 25952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:37.006845 25952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:37.006877 25952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:59:37.008193 25952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:38527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:37.008404 25952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:37.008649 25952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:37.013077 25958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:37.013064 25960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:37.013087 25957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:37.013274 25952 server_base.cc:1034] running on GCE node
I20260926 14:59:37.013470 25952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:37.013707 25952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:37.014851 25952 hybrid_clock.cc:648] HybridClock initialized: now 1790434777014835 us; error 27 us; skew 500 ppm
I20260926 14:59:37.016091 25952 webserver.cc:460] Webserver started at http://127.7.37.193:44843/ using document root <none> and password file <none>
I20260926 14:59:37.016311 25952 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:37.016361 25952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:37.016490 25952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:37.017473 25952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "48f40e2583644922b43629eb7f4ce109"
format_stamp: "Formatted at 2026-09-26 14:59:37 on dist-test-slave-f9xb"
server_key: "0c38504e7e728eabf196aa185822dd33"
server_key_iv: "417517ac8a9c6753a4f67cc0e7a349f5"
server_key_version: "encryptionkey@0"
I20260926 14:59:37.017803 25952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "48f40e2583644922b43629eb7f4ce109"
format_stamp: "Formatted at 2026-09-26 14:59:37 on dist-test-slave-f9xb"
server_key: "0c38504e7e728eabf196aa185822dd33"
server_key_iv: "417517ac8a9c6753a4f67cc0e7a349f5"
server_key_version: "encryptionkey@0"
I20260926 14:59:37.018966 25952 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 14:59:37.019580 25966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:37.019735 25952 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:37.019845 25952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "48f40e2583644922b43629eb7f4ce109"
format_stamp: "Formatted at 2026-09-26 14:59:37 on dist-test-slave-f9xb"
server_key: "0c38504e7e728eabf196aa185822dd33"
server_key_iv: "417517ac8a9c6753a4f67cc0e7a349f5"
server_key_version: "encryptionkey@0"
I20260926 14:59:37.019909 25952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:37.034269 25952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:37.034485 25952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:37.034595 25952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:37.034763 25952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:37.035028 25952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:59:37.035060 25952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:37.035084 25952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:59:37.035101 25952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:37.040230 25952 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:37737
I20260926 14:59:37.040266 26079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:37737 every 8 connection(s)
I20260926 14:59:37.040526 25952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:59:37.043885 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 25952
I20260926 14:59:37.043989 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:59:37.044171 7319 external_mini_cluster.cc:1428] Setting key 26127a645458a481dbbc80327208f719
I20260926 14:59:37.045030 25931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:37.041061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:47369 (local address 127.7.37.254:38527)
0926 14:59:37.041100 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:37.041102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:37.041322 (+ 220us) server_negotiation.cc:407] Connection header received
0926 14:59:37.041755 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:37.041756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:37.041776 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:37.041841 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:37.042417 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.042891 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.043805 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.043997 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.044053 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:37.044795 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:37.044797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:37.044797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:37.044802 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:37.044817 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:37.044822 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:37.044875 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:37.044928 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 14:59:37.044954 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 14:59:37.045027 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:38527
--builtin_ntp_servers=127.7.37.212:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 14:59:37.045154 26082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:37.041053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:38527 (local address 127.7.37.193:47369)
0926 14:59:37.041221 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:37.041249 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:37.041670 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:37.041898 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:37.041902 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:37.041911 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:37.042343 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:37.042347 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.042967 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.042968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:37.043705 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:37.043707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.044068 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.044069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:37.044188 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:37.044644 (+ 456us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:59:37.044645 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:37.044679 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:59:37.044704 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:37.044879 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:37.044882 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:37.044884 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:59:37.044926 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 14:59:37.044994 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260926 14:59:37.048096 26080 heartbeater.cc:347] Connected to a master server at 127.7.37.254:38527
I20260926 14:59:37.048226 26080 heartbeater.cc:464] Registering TS with master...
I20260926 14:59:37.048434 26080 heartbeater.cc:511] Master 127.7.37.254:38527 requested a full tablet report, sending...
I20260926 14:59:37.048993 25892 ts_manager.cc:194] Registered new tserver with Master: 48f40e2583644922b43629eb7f4ce109 (127.7.37.193:37737)
I20260926 14:59:37.049863 25892 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.37.193:47369
W20260926 14:59:37.121238 26083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:37.121410 26083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:37.121438 26083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:37.122558 26083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:37.122593 26083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:37.122627 26083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:59:37.123948 26083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:38527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:37.124150 26083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:37.124392 26083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:37.128675 26091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:37.128731 26089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:37.128783 26088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:37.128911 26083 server_base.cc:1034] running on GCE node
I20260926 14:59:37.129122 26083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:37.129325 26083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:37.130455 26083 hybrid_clock.cc:648] HybridClock initialized: now 1790434777130443 us; error 25 us; skew 500 ppm
I20260926 14:59:37.131488 26083 webserver.cc:460] Webserver started at http://127.7.37.194:45259/ using document root <none> and password file <none>
I20260926 14:59:37.131669 26083 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:37.131714 26083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:37.131860 26083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:37.132781 26083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "58f6ca1fa39d4e8fbaa57dd7bbeedc0d"
format_stamp: "Formatted at 2026-09-26 14:59:37 on dist-test-slave-f9xb"
server_key: "42eebc89d97c8b0aa330c0da2beab6d6"
server_key_iv: "913bb75f686f536de04f29610d6ac72d"
server_key_version: "encryptionkey@0"
I20260926 14:59:37.133069 26083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "58f6ca1fa39d4e8fbaa57dd7bbeedc0d"
format_stamp: "Formatted at 2026-09-26 14:59:37 on dist-test-slave-f9xb"
server_key: "42eebc89d97c8b0aa330c0da2beab6d6"
server_key_iv: "913bb75f686f536de04f29610d6ac72d"
server_key_version: "encryptionkey@0"
I20260926 14:59:37.134176 26083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:37.134814 26097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:37.134970 26083 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:37.135042 26083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "58f6ca1fa39d4e8fbaa57dd7bbeedc0d"
format_stamp: "Formatted at 2026-09-26 14:59:37 on dist-test-slave-f9xb"
server_key: "42eebc89d97c8b0aa330c0da2beab6d6"
server_key_iv: "913bb75f686f536de04f29610d6ac72d"
server_key_version: "encryptionkey@0"
I20260926 14:59:37.135097 26083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:37.153615 26083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:37.153824 26083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:37.153930 26083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:37.154100 26083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:37.154361 26083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:59:37.154392 26083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:37.154417 26083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:59:37.154438 26083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:37.159479 26083 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:43279
I20260926 14:59:37.159518 26210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:43279 every 8 connection(s)
I20260926 14:59:37.159788 26083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:59:37.163996 25931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:37.160348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:51005 (local address 127.7.37.254:38527)
0926 14:59:37.160417 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:37.160419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:37.160612 (+ 193us) server_negotiation.cc:407] Connection header received
0926 14:59:37.161007 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:37.161008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:37.161017 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:37.161062 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:37.161662 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.162130 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.162966 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.163152 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.163191 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:37.163817 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:37.163818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:37.163818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:37.163822 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:37.163835 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:37.163838 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:37.163874 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:37.163924 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 14:59:37.163942 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 14:59:37.164151 26213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:37.160344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:38527 (local address 127.7.37.194:51005)
0926 14:59:37.160510 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:37.160538 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:37.160937 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:37.161124 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:37.161128 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:37.161137 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:37.161595 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:37.161599 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.162185 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.162185 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:37.162912 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:37.162913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.163203 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.163203 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:37.163307 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:37.163680 (+ 373us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:59:37.163680 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:37.163703 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:59:37.163730 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:37.163879 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:37.163882 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:37.163884 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:59:37.163926 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 14:59:37.163993 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260926 14:59:37.164561 26211 heartbeater.cc:347] Connected to a master server at 127.7.37.254:38527
I20260926 14:59:37.164629 26211 heartbeater.cc:464] Registering TS with master...
I20260926 14:59:37.164750 26211 heartbeater.cc:511] Master 127.7.37.254:38527 requested a full tablet report, sending...
I20260926 14:59:37.165025 25892 ts_manager.cc:194] Registered new tserver with Master: 58f6ca1fa39d4e8fbaa57dd7bbeedc0d (127.7.37.194:43279)
I20260926 14:59:37.165340 25892 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.37.194:51005
I20260926 14:59:37.169483 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 26083
I20260926 14:59:37.169579 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:59:37.169746 7319 external_mini_cluster.cc:1428] Setting key 68c496a3f356a120891aeaf001c09cfc
I20260926 14:59:37.170480 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:38527
--builtin_ntp_servers=127.7.37.212:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 14:59:37.245209 26214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:37.245389 26214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:37.245416 26214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:37.246476 26214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:37.246512 26214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:37.246544 26214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:59:37.247798 26214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:38527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:37.247996 26214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:37.248230 26214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:37.252593 26220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:37.252684 26214 server_base.cc:1034] running on GCE node
W20260926 14:59:37.252622 26219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:37.252594 26222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:37.253021 26214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:37.253250 26214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:37.254377 26214 hybrid_clock.cc:648] HybridClock initialized: now 1790434777254365 us; error 24 us; skew 500 ppm
I20260926 14:59:37.255471 26214 webserver.cc:460] Webserver started at http://127.7.37.195:33069/ using document root <none> and password file <none>
I20260926 14:59:37.255672 26214 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:37.255717 26214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:37.255858 26214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:37.256806 26214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "ae8dec84e40142b7824c98bfaf8fdd7b"
format_stamp: "Formatted at 2026-09-26 14:59:37 on dist-test-slave-f9xb"
server_key: "fa0567d241d80bdac9de7fe5c60564fd"
server_key_iv: "f69cd36a51ec235aa190a43ce2a23638"
server_key_version: "encryptionkey@0"
I20260926 14:59:37.257102 26214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "ae8dec84e40142b7824c98bfaf8fdd7b"
format_stamp: "Formatted at 2026-09-26 14:59:37 on dist-test-slave-f9xb"
server_key: "fa0567d241d80bdac9de7fe5c60564fd"
server_key_iv: "f69cd36a51ec235aa190a43ce2a23638"
server_key_version: "encryptionkey@0"
I20260926 14:59:37.258168 26214 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:37.258764 26228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:37.258903 26214 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:37.258975 26214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "ae8dec84e40142b7824c98bfaf8fdd7b"
format_stamp: "Formatted at 2026-09-26 14:59:37 on dist-test-slave-f9xb"
server_key: "fa0567d241d80bdac9de7fe5c60564fd"
server_key_iv: "f69cd36a51ec235aa190a43ce2a23638"
server_key_version: "encryptionkey@0"
I20260926 14:59:37.259035 26214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:37.267118 26214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:37.267328 26214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:37.267427 26214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:37.267597 26214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:37.267880 26214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:59:37.267912 26214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:37.267937 26214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:59:37.267959 26214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:37.272946 26214 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:35465
I20260926 14:59:37.272980 26341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:35465 every 8 connection(s)
I20260926 14:59:37.273236 26214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:59:37.274176 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 26214
I20260926 14:59:37.274255 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:59:37.274423 7319 external_mini_cluster.cc:1428] Setting key d02f4df86bf221f0e3f455cfec2f4ed7
I20260926 14:59:37.277877 25931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:37.273807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:53095 (local address 127.7.37.254:38527)
0926 14:59:37.273862 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:37.273864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:37.274056 (+ 192us) server_negotiation.cc:407] Connection header received
0926 14:59:37.274502 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:37.274503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:37.274516 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:37.274580 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:37.275263 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.275769 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.276734 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.276922 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.276977 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:37.277632 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:37.277633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:37.277634 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:37.277638 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:37.277652 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:37.277656 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:37.277700 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:37.277775 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 14:59:37.277802 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 14:59:37.278055 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:37.273796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:38527 (local address 127.7.37.195:53095)
0926 14:59:37.273949 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:37.273983 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:37.274411 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:37.274647 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:37.274651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:37.274659 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:37.275186 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:37.275190 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.275847 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.275848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:37.276659 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:37.276662 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.276987 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.276988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:37.277111 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:37.277518 (+ 407us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:59:37.277519 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:37.277542 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:59:37.277561 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:37.277705 (+ 144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:37.277709 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:37.277712 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:59:37.277781 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 14:59:37.277870 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":61,"threads_started":1}
I20260926 14:59:37.278432 26342 heartbeater.cc:347] Connected to a master server at 127.7.37.254:38527
I20260926 14:59:37.278517 26342 heartbeater.cc:464] Registering TS with master...
I20260926 14:59:37.278688 26342 heartbeater.cc:511] Master 127.7.37.254:38527 requested a full tablet report, sending...
I20260926 14:59:37.278987 25892 ts_manager.cc:194] Registered new tserver with Master: ae8dec84e40142b7824c98bfaf8fdd7b (127.7.37.195:35465)
I20260926 14:59:37.279304 25892 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.37.195:53095
I20260926 14:59:37.286005 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 14:59:37.289737 25931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:37.287179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33940 (local address 127.7.37.254:38527)
0926 14:59:37.287234 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:37.287235 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:37.287303 (+ 68us) server_negotiation.cc:407] Connection header received
0926 14:59:37.287350 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:37.287350 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:37.287357 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:37.287390 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:37.287495 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.287967 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.288724 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.288897 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.288938 (+ 41us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:37.289553 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:37.289554 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:37.289554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:37.289557 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:37.289564 (+ 7us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:37.289567 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:37.289609 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:37.289667 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 14:59:37.289674 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 14:59:37.291154 25892 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:59:37.292652 25892 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:59:37.298270 26350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:37.294438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:43279 (local address 127.0.0.1:56710)
0926 14:59:37.294693 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:37.294702 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:37.294746 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:37.295004 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:37.295006 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:37.295009 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:37.295061 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:37.295062 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.296123 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.296124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:37.297203 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:37.297206 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.297989 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.297990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:37.298064 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:37.298076 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 14:59:37.298148 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 14:59:37.298198 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":1,"thread_start_us":63,"threads_started":1}
I20260926 14:59:37.298278 26354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:37.294603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56710 (local address 127.7.37.194:43279)
0926 14:59:37.294853 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:37.294857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:37.294866 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:59:37.294898 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:37.294899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:37.294914 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:37.294991 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:37.295162 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.296052 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.297306 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.297915 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.297981 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:37.298061 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:37.298162 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 14:59:37.298197 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":79,"threads_started":1}
I20260926 14:59:37.298550 26351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:37.294438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:37737 (local address 127.0.0.1:55068)
0926 14:59:37.294640 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:37.294677 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:37.294743 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:37.295421 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:37.295422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:37.295423 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:37.295463 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:37.295464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.296654 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.296655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:37.297431 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:37.297434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.298337 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.298339 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:37.298396 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:37.298400 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:59:37.298442 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 14:59:37.298481 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":44,"threads_started":1}
I20260926 14:59:37.298744 26353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:37.294595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:35465 (local address 127.0.0.1:34450)
0926 14:59:37.295065 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:37.295073 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:37.295113 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:37.295314 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:37.295314 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:37.295316 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:37.295354 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:37.295355 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.296592 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.296593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:37.297628 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:37.297630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.298500 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.298500 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:37.298553 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:37.298558 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 14:59:37.298652 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 14:59:37.298685 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":3,"thread_start_us":52,"threads_started":1}
I20260926 14:59:37.298765 26352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:37.294525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55068 (local address 127.7.37.193:37737)
0926 14:59:37.295228 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:37.295232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:37.295242 (+ 10us) server_negotiation.cc:407] Connection header received
0926 14:59:37.295277 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:37.295278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:37.295292 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:37.295357 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:37.295531 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.296598 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.297505 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.298161 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.298225 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:37.298302 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:37.298649 (+ 347us) server_negotiation.cc:299] Negotiation successful
0926 14:59:37.298690 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":110,"threads_started":1}
I20260926 14:59:37.299047 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:37.294819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34450 (local address 127.7.37.195:35465)
0926 14:59:37.294961 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:37.294965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:37.295122 (+ 157us) server_negotiation.cc:407] Connection header received
0926 14:59:37.295159 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:37.295162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:37.295175 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:37.295257 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:37.295417 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.296499 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.297706 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.298351 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.298404 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:37.298471 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:37.298943 (+ 472us) server_negotiation.cc:299] Negotiation successful
0926 14:59:37.298978 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
I20260926 14:59:37.298934 26145 tablet_service.cc:1467] Processing CreateTablet for tablet bb3c5f2597bd4bdc8d5371135dcdfc6c (DEFAULT_TABLE table=test-table [id=c6d11c39002940d690007427edf7a40c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:59:37.299283 26145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb3c5f2597bd4bdc8d5371135dcdfc6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:37.299505 26014 tablet_service.cc:1467] Processing CreateTablet for tablet bb3c5f2597bd4bdc8d5371135dcdfc6c (DEFAULT_TABLE table=test-table [id=c6d11c39002940d690007427edf7a40c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:59:37.299589 26276 tablet_service.cc:1467] Processing CreateTablet for tablet bb3c5f2597bd4bdc8d5371135dcdfc6c (DEFAULT_TABLE table=test-table [id=c6d11c39002940d690007427edf7a40c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:59:37.299850 26014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb3c5f2597bd4bdc8d5371135dcdfc6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:37.299909 26276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb3c5f2597bd4bdc8d5371135dcdfc6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:37.301995 26356 tablet_bootstrap.cc:492] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b: Bootstrap starting.
I20260926 14:59:37.302677 26356 tablet_bootstrap.cc:654] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:37.302731 26357 tablet_bootstrap.cc:492] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d: Bootstrap starting.
I20260926 14:59:37.303079 26356 log.cc:824] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:37.303462 26357 tablet_bootstrap.cc:654] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:37.303709 26357 log.cc:824] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:37.303800 26356 tablet_bootstrap.cc:492] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b: No bootstrap required, opened a new log
I20260926 14:59:37.303889 26356 ts_tablet_manager.cc:1397] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:59:37.303894 26358 tablet_bootstrap.cc:492] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 48f40e2583644922b43629eb7f4ce109: Bootstrap starting.
I20260926 14:59:37.304420 26357 tablet_bootstrap.cc:492] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d: No bootstrap required, opened a new log
I20260926 14:59:37.304436 26358 tablet_bootstrap.cc:654] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 48f40e2583644922b43629eb7f4ce109: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:37.304546 26357 ts_tablet_manager.cc:1397] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 14:59:37.304708 26358 log.cc:824] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 48f40e2583644922b43629eb7f4ce109: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:37.305449 26358 tablet_bootstrap.cc:492] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 48f40e2583644922b43629eb7f4ce109: No bootstrap required, opened a new log
I20260926 14:59:37.305511 26358 ts_tablet_manager.cc:1397] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 48f40e2583644922b43629eb7f4ce109: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:59:37.305938 26356 raft_consensus.cc:348] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f6ca1fa39d4e8fbaa57dd7bbeedc0d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43279 } } peers { permanent_uuid: "48f40e2583644922b43629eb7f4ce109" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37737 } } peers { permanent_uuid: "ae8dec84e40142b7824c98bfaf8fdd7b" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35465 } }
I20260926 14:59:37.306061 26356 raft_consensus.cc:374] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:37.306092 26356 raft_consensus.cc:729] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae8dec84e40142b7824c98bfaf8fdd7b, State: Initialized, Role: FOLLOWER
I20260926 14:59:37.306213 26356 consensus_queue.cc:260] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f6ca1fa39d4e8fbaa57dd7bbeedc0d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43279 } } peers { permanent_uuid: "48f40e2583644922b43629eb7f4ce109" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37737 } } peers { permanent_uuid: "ae8dec84e40142b7824c98bfaf8fdd7b" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35465 } }
I20260926 14:59:37.306445 26356 ts_tablet_manager.cc:1428] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 14:59:37.306551 26342 heartbeater.cc:503] Master 127.7.37.254:38527 was elected leader, sending a full tablet report...
I20260926 14:59:37.306857 26357 raft_consensus.cc:348] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f6ca1fa39d4e8fbaa57dd7bbeedc0d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43279 } } peers { permanent_uuid: "48f40e2583644922b43629eb7f4ce109" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37737 } } peers { permanent_uuid: "ae8dec84e40142b7824c98bfaf8fdd7b" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35465 } }
I20260926 14:59:37.307003 26357 raft_consensus.cc:374] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:37.307034 26357 raft_consensus.cc:729] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58f6ca1fa39d4e8fbaa57dd7bbeedc0d, State: Initialized, Role: FOLLOWER
I20260926 14:59:37.307179 26357 consensus_queue.cc:260] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f6ca1fa39d4e8fbaa57dd7bbeedc0d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43279 } } peers { permanent_uuid: "48f40e2583644922b43629eb7f4ce109" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37737 } } peers { permanent_uuid: "ae8dec84e40142b7824c98bfaf8fdd7b" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35465 } }
I20260926 14:59:37.307441 26357 ts_tablet_manager.cc:1428] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 14:59:37.307562 26211 heartbeater.cc:503] Master 127.7.37.254:38527 was elected leader, sending a full tablet report...
I20260926 14:59:37.307854 26358 raft_consensus.cc:348] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 48f40e2583644922b43629eb7f4ce109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f6ca1fa39d4e8fbaa57dd7bbeedc0d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43279 } } peers { permanent_uuid: "48f40e2583644922b43629eb7f4ce109" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37737 } } peers { permanent_uuid: "ae8dec84e40142b7824c98bfaf8fdd7b" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35465 } }
I20260926 14:59:37.307996 26358 raft_consensus.cc:374] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 48f40e2583644922b43629eb7f4ce109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:37.308059 26358 raft_consensus.cc:729] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 48f40e2583644922b43629eb7f4ce109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48f40e2583644922b43629eb7f4ce109, State: Initialized, Role: FOLLOWER
I20260926 14:59:37.308176 26358 consensus_queue.cc:260] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 48f40e2583644922b43629eb7f4ce109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f6ca1fa39d4e8fbaa57dd7bbeedc0d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43279 } } peers { permanent_uuid: "48f40e2583644922b43629eb7f4ce109" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37737 } } peers { permanent_uuid: "ae8dec84e40142b7824c98bfaf8fdd7b" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35465 } }
I20260926 14:59:37.308406 26358 ts_tablet_manager.cc:1428] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 48f40e2583644922b43629eb7f4ce109: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 14:59:37.308491 26080 heartbeater.cc:503] Master 127.7.37.254:38527 was elected leader, sending a full tablet report...
I20260926 14:59:37.315680 26362 raft_consensus.cc:482] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:59:37.315769 26362 raft_consensus.cc:504] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f6ca1fa39d4e8fbaa57dd7bbeedc0d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43279 } } peers { permanent_uuid: "48f40e2583644922b43629eb7f4ce109" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37737 } } peers { permanent_uuid: "ae8dec84e40142b7824c98bfaf8fdd7b" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35465 } }
I20260926 14:59:37.316030 26362 leader_election.cc:288] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58f6ca1fa39d4e8fbaa57dd7bbeedc0d (127.7.37.194:43279), 48f40e2583644922b43629eb7f4ce109 (127.7.37.193:37737)
I20260926 14:59:37.318264 26352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:37.316211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:49753 (local address 127.7.37.193:37737)
0926 14:59:37.316254 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:37.316256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:37.316273 (+ 17us) server_negotiation.cc:407] Connection header received
0926 14:59:37.316290 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:37.316291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:37.316299 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:37.316337 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:37.316438 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.316855 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.317666 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.317994 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.318050 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:37.318055 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:37.318170 (+ 115us) server_negotiation.cc:299] Negotiation successful
0926 14:59:37.318202 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 14:59:37.318310 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:37.316147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:37737 (local address 127.7.37.195:49753)
0926 14:59:37.316191 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:37.316200 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:37.316235 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:37.316339 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:37.316339 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:37.316340 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:37.316386 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:37.316388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.316914 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.316915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:37.317594 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:37.317596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.318057 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.318058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:37.318116 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:37.318119 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:59:37.318190 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 14:59:37.318224 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 14:59:37.318611 26034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb3c5f2597bd4bdc8d5371135dcdfc6c" candidate_uuid: "ae8dec84e40142b7824c98bfaf8fdd7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48f40e2583644922b43629eb7f4ce109" is_pre_election: true
I20260926 14:59:37.318743 26354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:37.316218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:43159 (local address 127.7.37.194:43279)
0926 14:59:37.316254 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:37.316256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:37.316370 (+ 114us) server_negotiation.cc:407] Connection header received
0926 14:59:37.316432 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:37.316433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:37.316441 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:37.316485 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:37.316636 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.317055 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.317931 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.318341 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.318413 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:37.318419 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:37.318651 (+ 232us) server_negotiation.cc:299] Negotiation successful
0926 14:59:37.318690 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:59:37.318746 26365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:37.316173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:43279 (local address 127.7.37.195:43159)
0926 14:59:37.316303 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:37.316312 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:37.316371 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:37.316504 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:37.316504 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:37.316506 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:37.316581 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:37.316583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.317133 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.317134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:37.317853 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:37.317855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.318466 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.318467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:37.318533 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:37.318538 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 14:59:37.318626 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 14:59:37.318662 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260926 14:59:37.318778 26034 raft_consensus.cc:2454] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 48f40e2583644922b43629eb7f4ce109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae8dec84e40142b7824c98bfaf8fdd7b in term 0.
I20260926 14:59:37.319037 26230 leader_election.cc:302] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48f40e2583644922b43629eb7f4ce109, ae8dec84e40142b7824c98bfaf8fdd7b; no voters:
I20260926 14:59:37.319205 26165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb3c5f2597bd4bdc8d5371135dcdfc6c" candidate_uuid: "ae8dec84e40142b7824c98bfaf8fdd7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f6ca1fa39d4e8fbaa57dd7bbeedc0d" is_pre_election: true
I20260926 14:59:37.319257 26362 raft_consensus.cc:2781] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:59:37.319300 26165 raft_consensus.cc:2454] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae8dec84e40142b7824c98bfaf8fdd7b in term 0.
I20260926 14:59:37.319325 26362 raft_consensus.cc:482] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:59:37.319358 26362 raft_consensus.cc:3037] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:37.320006 26362 raft_consensus.cc:504] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f6ca1fa39d4e8fbaa57dd7bbeedc0d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43279 } } peers { permanent_uuid: "48f40e2583644922b43629eb7f4ce109" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37737 } } peers { permanent_uuid: "ae8dec84e40142b7824c98bfaf8fdd7b" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35465 } }
I20260926 14:59:37.320118 26362 leader_election.cc:288] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [CANDIDATE]: Term 1 election: Requested vote from peers 58f6ca1fa39d4e8fbaa57dd7bbeedc0d (127.7.37.194:43279), 48f40e2583644922b43629eb7f4ce109 (127.7.37.193:37737)
I20260926 14:59:37.320281 26034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb3c5f2597bd4bdc8d5371135dcdfc6c" candidate_uuid: "ae8dec84e40142b7824c98bfaf8fdd7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48f40e2583644922b43629eb7f4ce109"
I20260926 14:59:37.320269 26165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb3c5f2597bd4bdc8d5371135dcdfc6c" candidate_uuid: "ae8dec84e40142b7824c98bfaf8fdd7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f6ca1fa39d4e8fbaa57dd7bbeedc0d"
I20260926 14:59:37.320343 26165 raft_consensus.cc:3037] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:37.320343 26034 raft_consensus.cc:3037] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 48f40e2583644922b43629eb7f4ce109 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:37.321067 26165 raft_consensus.cc:2454] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae8dec84e40142b7824c98bfaf8fdd7b in term 1.
I20260926 14:59:37.321067 26034 raft_consensus.cc:2454] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 48f40e2583644922b43629eb7f4ce109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae8dec84e40142b7824c98bfaf8fdd7b in term 1.
I20260926 14:59:37.321261 26232 leader_election.cc:302] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58f6ca1fa39d4e8fbaa57dd7bbeedc0d, ae8dec84e40142b7824c98bfaf8fdd7b; no voters:
I20260926 14:59:37.321336 26362 raft_consensus.cc:2781] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:37.321494 26362 raft_consensus.cc:686] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [term 1 LEADER]: Becoming Leader. State: Replica: ae8dec84e40142b7824c98bfaf8fdd7b, State: Running, Role: LEADER
I20260926 14:59:37.321573 26362 consensus_queue.cc:237] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f6ca1fa39d4e8fbaa57dd7bbeedc0d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43279 } } peers { permanent_uuid: "48f40e2583644922b43629eb7f4ce109" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37737 } } peers { permanent_uuid: "ae8dec84e40142b7824c98bfaf8fdd7b" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35465 } }
I20260926 14:59:37.322183 25892 catalog_manager.cc:5495] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b reported cstate change: term changed from 0 to 1, leader changed from <none> to ae8dec84e40142b7824c98bfaf8fdd7b (127.7.37.195). New cstate: current_term: 1 leader_uuid: "ae8dec84e40142b7824c98bfaf8fdd7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f6ca1fa39d4e8fbaa57dd7bbeedc0d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48f40e2583644922b43629eb7f4ce109" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae8dec84e40142b7824c98bfaf8fdd7b" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35465 } health_report { overall_health: HEALTHY } } }
I20260926 14:59:37.332860 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:37.330938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34462 (local address 127.7.37.195:35465)
0926 14:59:37.330972 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:37.330974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:37.330983 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:59:37.331041 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:37.331041 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:37.331055 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:37.331105 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:59:37.331199 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.331604 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.332323 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:37.332494 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:37.332543 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:37.332637 (+ 94us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:59:37.332709 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:59:37.332768 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:37.332799 (+ 31us) server_negotiation.cc:299] Negotiation successful
0926 14:59:37.332805 (+ 6us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 14:59:37.334131 26165 raft_consensus.cc:1261] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d [term 1 FOLLOWER]: Refusing update from remote peer ae8dec84e40142b7824c98bfaf8fdd7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:59:37.334142 26034 raft_consensus.cc:1261] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 48f40e2583644922b43629eb7f4ce109 [term 1 FOLLOWER]: Refusing update from remote peer ae8dec84e40142b7824c98bfaf8fdd7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:59:37.334440 26366 consensus_queue.cc:1035] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f6ca1fa39d4e8fbaa57dd7bbeedc0d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 43279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:59:37.334544 26362 consensus_queue.cc:1035] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "48f40e2583644922b43629eb7f4ce109" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 37737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:59:37.336480 26369 mvcc.cc:205] Tried to move back new op lower bound from 7333620847958872064 to 7333620847909756928. Current Snapshot: MvccSnapshot[applied={T|T < 7333620847958872064}]
I20260926 14:59:37.336853 26370 mvcc.cc:205] Tried to move back new op lower bound from 7333620847958872064 to 7333620847909756928. Current Snapshot: MvccSnapshot[applied={T|T < 7333620847958872064}]
I20260926 14:59:37.338713 25890 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:59:37.338785 25890 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:59:37.339353 25890 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b31bf6cb47cc46c48e16f2710964e6a7: Sending DeleteTablet for 3 replicas of tablet bb3c5f2597bd4bdc8d5371135dcdfc6c
I20260926 14:59:37.339628 26145 tablet_service.cc:1514] Processing DeleteTablet for tablet bb3c5f2597bd4bdc8d5371135dcdfc6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:59:37 UTC) from {username='slave'} at 127.0.0.1:56710
I20260926 14:59:37.339672 26276 tablet_service.cc:1514] Processing DeleteTablet for tablet bb3c5f2597bd4bdc8d5371135dcdfc6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:59:37 UTC) from {username='slave'} at 127.0.0.1:34450
I20260926 14:59:37.339859 26379 tablet_replica.cc:330] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d: stopping tablet replica
I20260926 14:59:37.339847 26014 tablet_service.cc:1514] Processing DeleteTablet for tablet bb3c5f2597bd4bdc8d5371135dcdfc6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:59:37 UTC) from {username='slave'} at 127.0.0.1:55068
I20260926 14:59:37.339934 26379 raft_consensus.cc:2229] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:59:37.339941 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 25952
I20260926 14:59:37.340019 26379 raft_consensus.cc:2258] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:59:37.341082 26379 ts_tablet_manager.cc:1905] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:59:37.342423 26379 ts_tablet_manager.cc:1918] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:59:37.342483 26379 log.cc:1196] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/bb3c5f2597bd4bdc8d5371135dcdfc6c
I20260926 14:59:37.342716 26379 ts_tablet_manager.cc:1939] T bb3c5f2597bd4bdc8d5371135dcdfc6c P 58f6ca1fa39d4e8fbaa57dd7bbeedc0d: Deleting consensus metadata
I20260926 14:59:37.343030 25879 catalog_manager.cc:4841] TS 58f6ca1fa39d4e8fbaa57dd7bbeedc0d (127.7.37.194:43279): tablet bb3c5f2597bd4bdc8d5371135dcdfc6c (table test-table [id=c6d11c39002940d690007427edf7a40c]) successfully deleted
I20260926 14:59:37.343843 26380 tablet_replica.cc:330] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b: stopping tablet replica
I20260926 14:59:37.343926 26380 raft_consensus.cc:2229] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:59:37.344030 26380 raft_consensus.cc:2258] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:59:37.344377 26380 ts_tablet_manager.cc:1905] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:59:37.345508 26380 ts_tablet_manager.cc:1918] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:59:37.345619 26380 log.cc:1196] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/bb3c5f2597bd4bdc8d5371135dcdfc6c
W20260926 14:59:37.345697 25877 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.37.193:37737 (error 108)
I20260926 14:59:37.345893 26380 ts_tablet_manager.cc:1939] T bb3c5f2597bd4bdc8d5371135dcdfc6c P ae8dec84e40142b7824c98bfaf8fdd7b: Deleting consensus metadata
I20260926 14:59:37.345924 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 26083
I20260926 14:59:37.346896 25877 catalog_manager.cc:4841] TS ae8dec84e40142b7824c98bfaf8fdd7b (127.7.37.195:35465): tablet bb3c5f2597bd4bdc8d5371135dcdfc6c (table test-table [id=c6d11c39002940d690007427edf7a40c]) successfully deleted
I20260926 14:59:37.351441 26353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:37.351341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:37737 (local address 127.0.0.1:55078)
0926 14:59:37.351386 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:37.351416 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.193:37737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 14:59:37.351500 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 26214
W20260926 14:59:37.354995 25877 catalog_manager.cc:4584] TS 48f40e2583644922b43629eb7f4ce109 (127.7.37.193:37737): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bb3c5f2597bd4bdc8d5371135dcdfc6c: Network error: Client connection negotiation failed: client connection to 127.7.37.193:37737: connect: Connection refused (error 111)
I20260926 14:59:37.355213 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 25861
2026-09-26T14:59:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (608 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:37 dist-test-slave-f9xb krb5kdc[26385](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:37 dist-test-slave-f9xb krb5kdc[26385](info): set up 2 sockets
Sep 26 14:59:37 dist-test-slave-f9xb krb5kdc[26385](info): commencing operation
krb5kdc: starting...
W20260926 14:59:38.291242 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.919s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:38 dist-test-slave-f9xb krb5kdc[26385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:59:38.392283 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:33149
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:39275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:33149
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:38.466445 26398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:38.466612 26398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:38.466632 26398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:38.467674 26398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:38.467707 26398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:38.467720 26398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:38.467730 26398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:38.467778 26398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:38.469153 26398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:39275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:33149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:33149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:38.469323 26398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:38.469471 26398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:38.473663 26403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:38.473770 26404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:38.473825 26398 server_base.cc:1034] running on GCE node
W20260926 14:59:38.473688 26406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:38.474327 26398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:38.474541 26398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:38.475663 26398 hybrid_clock.cc:648] HybridClock initialized: now 1790434778475651 us; error 27 us; skew 500 ppm
Sep 26 14:59:38 dist-test-slave-f9xb krb5kdc[26385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434778, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:38.477789 26398 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:59:38.478163 26398 webserver.cc:460] Webserver started at http://127.7.37.254:46815/ using document root <none> and password file <none>
I20260926 14:59:38.478325 26398 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:38.478361 26398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:38.478442 26398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:38.479303 26398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "72b32861e1be4283b806d9057aba77ca"
format_stamp: "Formatted at 2026-09-26 14:59:38 on dist-test-slave-f9xb"
server_key: "eccc9adf1fb13abe9721c818c83ecd02"
server_key_iv: "72057dc171b864fc7a4439c4aed4894e"
server_key_version: "encryptionkey@0"
I20260926 14:59:38.479559 26398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "72b32861e1be4283b806d9057aba77ca"
format_stamp: "Formatted at 2026-09-26 14:59:38 on dist-test-slave-f9xb"
server_key: "eccc9adf1fb13abe9721c818c83ecd02"
server_key_iv: "72057dc171b864fc7a4439c4aed4894e"
server_key_version: "encryptionkey@0"
I20260926 14:59:38.480719 26398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:38.481348 26413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:38.481488 26398 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:38.481563 26398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "72b32861e1be4283b806d9057aba77ca"
format_stamp: "Formatted at 2026-09-26 14:59:38 on dist-test-slave-f9xb"
server_key: "eccc9adf1fb13abe9721c818c83ecd02"
server_key_iv: "72057dc171b864fc7a4439c4aed4894e"
server_key_version: "encryptionkey@0"
I20260926 14:59:38.481616 26398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:38.512955 26398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:38.514417 26398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:38.514508 26398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:38.517774 26398 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:33149
I20260926 14:59:38.517817 26465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:33149 every 8 connection(s)
I20260926 14:59:38.518061 26398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:38.518790 26466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:38.520681 26466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca: Bootstrap starting.
I20260926 14:59:38.521180 26466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:38.521420 26466 log.cc:824] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:38.521992 26466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca: No bootstrap required, opened a new log
I20260926 14:59:38.523447 26466 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b32861e1be4283b806d9057aba77ca" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 33149 } }
I20260926 14:59:38.523566 26466 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:38.523595 26466 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72b32861e1be4283b806d9057aba77ca, State: Initialized, Role: FOLLOWER
I20260926 14:59:38.523708 26466 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b32861e1be4283b806d9057aba77ca" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 33149 } }
I20260926 14:59:38.523808 26466 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:38.523846 26466 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:38.523891 26466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:38.524335 26466 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b32861e1be4283b806d9057aba77ca" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 33149 } }
I20260926 14:59:38.524420 26466 leader_election.cc:302] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72b32861e1be4283b806d9057aba77ca; no voters:
I20260926 14:59:38.524562 26466 leader_election.cc:288] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:38.524585 26469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:38.524719 26469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca [term 1 LEADER]: Becoming Leader. State: Replica: 72b32861e1be4283b806d9057aba77ca, State: Running, Role: LEADER
I20260926 14:59:38.524780 26466 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:38.524799 26469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b32861e1be4283b806d9057aba77ca" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 33149 } }
I20260926 14:59:38.525110 26471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72b32861e1be4283b806d9057aba77ca. Latest consensus state: current_term: 1 leader_uuid: "72b32861e1be4283b806d9057aba77ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b32861e1be4283b806d9057aba77ca" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 33149 } } }
I20260926 14:59:38.525184 26471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:38.525234 26470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72b32861e1be4283b806d9057aba77ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b32861e1be4283b806d9057aba77ca" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 33149 } } }
I20260926 14:59:38.525280 26470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:38.525388 26476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:38.525820 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 26398
I20260926 14:59:38.525839 26476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:38.525920 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:38.526093 7319 external_mini_cluster.cc:1428] Setting key c6e6b0f5359b1094bd0be232e214e728
I20260926 14:59:38.527197 26476 catalog_manager.cc:1347] Generated new cluster ID: 8f9531645f3e4b8086b6afc984fd88fc
I20260926 14:59:38.527244 26476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 14:59:38 dist-test-slave-f9xb krb5kdc[26385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:59:38.541389 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:38.526574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39686 (local address 127.7.37.254:33149)
0926 14:59:38.526848 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:38.526856 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:38.526881 (+ 25us) server_negotiation.cc:407] Connection header received
0926 14:59:38.526983 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:38.526989 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:38.527049 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:38.531856 (+ 4807us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:38.532590 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:38.533911 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:38.534960 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:38.535508 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:38.535589 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:38.538060 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:38.538064 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:38.538064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:38.538075 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:38.539839 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:38.540163 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:38.540166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:38.540166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:38.540272 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:38.540433 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:38.540434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:38.540434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:38.540713 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:38.540852 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:38.540943 (+ 91us) server_negotiation.cc:299] Negotiation successful
0926 14:59:38.541165 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":58,"threads_started":1}
I20260926 14:59:38.541803 26476 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:38.542235 26476 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:38.552155 26476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca: Generated new TSK 0
I20260926 14:59:38.552317 26476 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.37.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
Entry for principal kudu/127.7.37.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab.
I20260926 14:59:38.581851 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:33149
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:39275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:38.676115 26492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:38.676292 26492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:38.676321 26492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:38.677409 26492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:38.677445 26492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:38.677479 26492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:59:38.678860 26492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:39275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.193.keytab
--principal=kudu/127.7.37.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:33149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:38.679066 26492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:38.679245 26492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:38.683612 26497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:38.683622 26498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:38.683622 26500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:38.683836 26492 server_base.cc:1034] running on GCE node
I20260926 14:59:38.684011 26492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:38.684245 26492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:38.685388 26492 hybrid_clock.cc:648] HybridClock initialized: now 1790434778685367 us; error 33 us; skew 500 ppm
Sep 26 14:59:38 dist-test-slave-f9xb krb5kdc[26385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434778, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:38.687583 26492 init.cc:377] Logged in from keytab as kudu/127.7.37.193@KRBTEST.COM (short username kudu)
I20260926 14:59:38.687963 26492 webserver.cc:460] Webserver started at http://127.7.37.193:36789/ using document root <none> and password file <none>
I20260926 14:59:38.688117 26492 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:38.688153 26492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:38.688243 26492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:38.689105 26492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "d8059e855aeb42e295c46af2988f3bab"
format_stamp: "Formatted at 2026-09-26 14:59:38 on dist-test-slave-f9xb"
server_key: "8866d5a12142523724d51371bf8fe85c"
server_key_iv: "74a425c94a345a782d92b4273eaecfc3"
server_key_version: "encryptionkey@0"
I20260926 14:59:38.689374 26492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "d8059e855aeb42e295c46af2988f3bab"
format_stamp: "Formatted at 2026-09-26 14:59:38 on dist-test-slave-f9xb"
server_key: "8866d5a12142523724d51371bf8fe85c"
server_key_iv: "74a425c94a345a782d92b4273eaecfc3"
server_key_version: "encryptionkey@0"
I20260926 14:59:38.690487 26492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:38.691072 26507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:38.691228 26492 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:38.691304 26492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "d8059e855aeb42e295c46af2988f3bab"
format_stamp: "Formatted at 2026-09-26 14:59:38 on dist-test-slave-f9xb"
server_key: "8866d5a12142523724d51371bf8fe85c"
server_key_iv: "74a425c94a345a782d92b4273eaecfc3"
server_key_version: "encryptionkey@0"
I20260926 14:59:38.691360 26492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:38.698726 26492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:38.700567 26492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:38.700681 26492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:38.700863 26492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:38.701097 26492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:59:38.701128 26492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:38.701151 26492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:59:38.701174 26492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:38.706389 26492 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:39505
I20260926 14:59:38.706424 26620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:39505 every 8 connection(s)
I20260926 14:59:38.706670 26492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
Sep 26 14:59:38 dist-test-slave-f9xb krb5kdc[26385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434778, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.193@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:59:38.714067 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:38.707218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:48099 (local address 127.7.37.254:33149)
0926 14:59:38.707290 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:38.707292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:38.707453 (+ 161us) server_negotiation.cc:407] Connection header received
0926 14:59:38.707880 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:38.707881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:38.707890 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:38.707935 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:38.708693 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:38.709175 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:38.710071 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:38.710247 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:38.710291 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:38.712108 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:38.712109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:38.712109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:38.712114 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:38.713132 (+ 1018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:38.713441 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:38.713442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:38.713442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:38.713481 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:38.713640 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:38.713641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:38.713641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:38.713726 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:38.713776 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:38.713954 (+ 178us) server_negotiation.cc:299] Negotiation successful
0926 14:59:38.714009 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 14:59:38.714210 26623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:38.707199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:33149 (local address 127.7.37.193:48099)
0926 14:59:38.707354 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:38.707384 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:38.707801 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:38.707983 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:38.707986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:38.708462 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:38.708630 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:38.708634 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:38.709249 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:38.709250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:38.710012 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:38.710014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:38.710297 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:38.710297 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:38.710380 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:38.710789 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:59:38.710797 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:38.712030 (+ 1233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:38.713195 (+ 1165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:38.713201 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:38.713202 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:38.713389 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:38.713523 (+ 134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:38.713523 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:38.713523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:38.713602 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:38.713769 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:38.713772 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:38.713866 (+ 94us) client_negotiation.cc:769] Sending connection context
0926 14:59:38.713936 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 14:59:38.714035 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260926 14:59:38.714615 26621 heartbeater.cc:347] Connected to a master server at 127.7.37.254:33149
I20260926 14:59:38.714694 26621 heartbeater.cc:464] Registering TS with master...
I20260926 14:59:38.714846 26621 heartbeater.cc:511] Master 127.7.37.254:33149 requested a full tablet report, sending...
I20260926 14:59:38.715269 26430 ts_manager.cc:194] Registered new tserver with Master: d8059e855aeb42e295c46af2988f3bab (127.7.37.193:39505)
I20260926 14:59:38.715907 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 26492
I20260926 14:59:38.715916 26430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.193@KRBTEST.COM'} at 127.7.37.193:48099
I20260926 14:59:38.716018 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:59:38.716229 7319 external_mini_cluster.cc:1428] Setting key a24cff8b0b68781d0eff395b95a5c276
WARNING: no policy specified for kudu/127.7.37.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
Entry for principal kudu/127.7.37.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab.
I20260926 14:59:38.736459 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:33149
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:39275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:38.810549 26626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:38.810714 26626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:38.810735 26626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:38.811836 26626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:38.811872 26626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:38.811898 26626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:59:38.813261 26626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:39275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.194.keytab
--principal=kudu/127.7.37.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:33149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:38.813442 26626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:38.813624 26626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:38.817994 26634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:38.818028 26631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:38.818063 26632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:38.818218 26626 server_base.cc:1034] running on GCE node
I20260926 14:59:38.818413 26626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:38.818581 26626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:38.819713 26626 hybrid_clock.cc:648] HybridClock initialized: now 1790434778819698 us; error 28 us; skew 500 ppm
Sep 26 14:59:38 dist-test-slave-f9xb krb5kdc[26385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434778, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:38.822065 26626 init.cc:377] Logged in from keytab as kudu/127.7.37.194@KRBTEST.COM (short username kudu)
I20260926 14:59:38.822453 26626 webserver.cc:460] Webserver started at http://127.7.37.194:43631/ using document root <none> and password file <none>
I20260926 14:59:38.822625 26626 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:38.822669 26626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:38.822784 26626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:38.823668 26626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "ee44fd027ec545899b90f629be6ff81d"
format_stamp: "Formatted at 2026-09-26 14:59:38 on dist-test-slave-f9xb"
server_key: "7f3c22e44d2a7c52a17da279753d224a"
server_key_iv: "7aad621634a60d1976254c9b149e5979"
server_key_version: "encryptionkey@0"
I20260926 14:59:38.824012 26626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "ee44fd027ec545899b90f629be6ff81d"
format_stamp: "Formatted at 2026-09-26 14:59:38 on dist-test-slave-f9xb"
server_key: "7f3c22e44d2a7c52a17da279753d224a"
server_key_iv: "7aad621634a60d1976254c9b149e5979"
server_key_version: "encryptionkey@0"
I20260926 14:59:38.825145 26626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 14:59:38.825790 26641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:38.825934 26626 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:38.826009 26626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "ee44fd027ec545899b90f629be6ff81d"
format_stamp: "Formatted at 2026-09-26 14:59:38 on dist-test-slave-f9xb"
server_key: "7f3c22e44d2a7c52a17da279753d224a"
server_key_iv: "7aad621634a60d1976254c9b149e5979"
server_key_version: "encryptionkey@0"
I20260926 14:59:38.826067 26626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:38.840611 26626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:38.842451 26626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:38.842571 26626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:38.842741 26626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:38.843045 26626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:59:38.843078 26626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:38.843103 26626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:59:38.843127 26626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:38.848061 26626 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:42993
I20260926 14:59:38.848116 26754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:42993 every 8 connection(s)
I20260926 14:59:38.848351 26626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:59:38.849900 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 26626
I20260926 14:59:38.850003 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:59:38.850188 7319 external_mini_cluster.cc:1428] Setting key 551608ce670056788b5788535f170860
Sep 26 14:59:38 dist-test-slave-f9xb krb5kdc[26385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434778, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.194@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:59:38.857200 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:38.848936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:35117 (local address 127.7.37.254:33149)
0926 14:59:38.849002 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:38.849004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:38.849164 (+ 160us) server_negotiation.cc:407] Connection header received
0926 14:59:38.849552 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:38.849553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:38.849570 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:38.849636 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:38.850594 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:38.851239 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:38.852364 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:38.852552 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:38.852617 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:38.854682 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:38.854683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:38.854684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:38.854691 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:38.855964 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:38.856341 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:38.856342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:38.856343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:38.856397 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:38.856618 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:38.856619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:38.856620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:38.856725 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:38.856785 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:38.857024 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 14:59:38.857110 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
WARNING: no policy specified for kudu/127.7.37.195@KRBTEST.COM; defaulting to no policy
I20260926 14:59:38.857339 26757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:38.848875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:33149 (local address 127.7.37.194:35117)
0926 14:59:38.849061 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:38.849097 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:38.849480 (+ 383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:38.849680 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:38.849683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:38.850276 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:38.850511 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:38.850516 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:38.851324 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:38.851324 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:38.852283 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:38.852286 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:38.852626 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:38.852627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:38.852755 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:38.853206 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:59:38.853216 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:38.854593 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:38.856035 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:38.856042 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:38.856044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:38.856262 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:38.856456 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:38.856456 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:38.856457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:38.856558 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:38.856782 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:38.856788 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:38.856911 (+ 123us) client_negotiation.cc:769] Sending connection context
0926 14:59:38.856997 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 14:59:38.857147 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":78,"threads_started":1}
I20260926 14:59:38.857936 26755 heartbeater.cc:347] Connected to a master server at 127.7.37.254:33149
I20260926 14:59:38.858016 26755 heartbeater.cc:464] Registering TS with master...
I20260926 14:59:38.858207 26755 heartbeater.cc:511] Master 127.7.37.254:33149 requested a full tablet report, sending...
I20260926 14:59:38.858525 26430 ts_manager.cc:194] Registered new tserver with Master: ee44fd027ec545899b90f629be6ff81d (127.7.37.194:42993)
I20260926 14:59:38.858870 26430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.194@KRBTEST.COM'} at 127.7.37.194:35117
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
Entry for principal kudu/127.7.37.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab.
I20260926 14:59:38.873636 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:33149
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:39275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:38.948464 26760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:38.948630 26760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:38.948652 26760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:38.949903 26760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:38.949949 26760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:38.949975 26760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:59:38.951467 26760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:39275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.195.keytab
--principal=kudu/127.7.37.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:33149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:38.951653 26760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:38.951864 26760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:38.956252 26765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:38.956269 26766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:38.956307 26768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:38.956458 26760 server_base.cc:1034] running on GCE node
I20260926 14:59:38.956624 26760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:38.956839 26760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:38.957991 26760 hybrid_clock.cc:648] HybridClock initialized: now 1790434778957977 us; error 30 us; skew 500 ppm
Sep 26 14:59:38 dist-test-slave-f9xb krb5kdc[26385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434778, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:38.960177 26760 init.cc:377] Logged in from keytab as kudu/127.7.37.195@KRBTEST.COM (short username kudu)
I20260926 14:59:38.960570 26760 webserver.cc:460] Webserver started at http://127.7.37.195:38775/ using document root <none> and password file <none>
I20260926 14:59:38.960758 26760 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:38.960803 26760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:38.960913 26760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:38.961802 26760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "5681eb6fd2374a95a6bbf0f47fbd98b7"
format_stamp: "Formatted at 2026-09-26 14:59:38 on dist-test-slave-f9xb"
server_key: "db59a20d2abc39b0116fe44a8f25614b"
server_key_iv: "83e5c31b45ef78ff037fb230afe2ee6b"
server_key_version: "encryptionkey@0"
I20260926 14:59:38.962093 26760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "5681eb6fd2374a95a6bbf0f47fbd98b7"
format_stamp: "Formatted at 2026-09-26 14:59:38 on dist-test-slave-f9xb"
server_key: "db59a20d2abc39b0116fe44a8f25614b"
server_key_iv: "83e5c31b45ef78ff037fb230afe2ee6b"
server_key_version: "encryptionkey@0"
I20260926 14:59:38.963169 26760 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:38.963855 26775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:38.963984 26760 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:38.964064 26760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "5681eb6fd2374a95a6bbf0f47fbd98b7"
format_stamp: "Formatted at 2026-09-26 14:59:38 on dist-test-slave-f9xb"
server_key: "db59a20d2abc39b0116fe44a8f25614b"
server_key_iv: "83e5c31b45ef78ff037fb230afe2ee6b"
server_key_version: "encryptionkey@0"
I20260926 14:59:38.964128 26760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:38.976687 26760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:38.978191 26760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:38.978307 26760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:38.978484 26760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:38.978725 26760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:59:38.978755 26760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:38.978780 26760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:59:38.978796 26760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:38.984210 26760 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:41509
I20260926 14:59:38.984284 26888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:41509 every 8 connection(s)
I20260926 14:59:38.984522 26760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:59:38.987259 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 26760
I20260926 14:59:38.987368 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:59:38.987528 7319 external_mini_cluster.cc:1428] Setting key f17388270096139a3b45ce60a50f4b61
Sep 26 14:59:38 dist-test-slave-f9xb krb5kdc[26385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434778, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.195@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:59:38.992467 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:38.985113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:60473 (local address 127.7.37.254:33149)
0926 14:59:38.985168 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:38.985170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:38.985332 (+ 162us) server_negotiation.cc:407] Connection header received
0926 14:59:38.985758 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:38.985759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:38.985770 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:38.985819 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:38.986607 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:38.987080 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:38.988295 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:38.988520 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:38.988571 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:38.990302 (+ 1731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:38.990303 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:38.990303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:38.990310 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:38.991445 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:38.991795 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:38.991797 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:38.991797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:38.991850 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:38.992032 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:38.992032 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:38.992032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:38.992126 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:38.992169 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:38.992352 (+ 183us) server_negotiation.cc:299] Negotiation successful
0926 14:59:38.992409 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 14:59:38.992594 26891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:38.985089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:33149 (local address 127.7.37.195:60473)
0926 14:59:38.985235 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:38.985265 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:38.985672 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:38.985875 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:38.985878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:38.986361 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:38.986522 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:38.986526 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:38.987153 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:38.987154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:38.988224 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:38.988226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:38.988577 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:38.988578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:38.988676 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:38.989026 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 14:59:38.989033 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:38.990213 (+ 1180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:38.991515 (+ 1302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:38.991520 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:38.991521 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:38.991719 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:38.991899 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 14:59:38.991900 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 14:59:38.991900 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 14:59:38.991987 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 14:59:38.992166 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:38.992169 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:38.992260 (+ 91us) client_negotiation.cc:769] Sending connection context
0926 14:59:38.992328 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 14:59:38.992431 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260926 14:59:38.992972 26889 heartbeater.cc:347] Connected to a master server at 127.7.37.254:33149
I20260926 14:59:38.993044 26889 heartbeater.cc:464] Registering TS with master...
I20260926 14:59:38.993188 26889 heartbeater.cc:511] Master 127.7.37.254:33149 requested a full tablet report, sending...
I20260926 14:59:38.993494 26430 ts_manager.cc:194] Registered new tserver with Master: 5681eb6fd2374a95a6bbf0f47fbd98b7 (127.7.37.195:41509)
I20260926 14:59:38.993809 26430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.37.195@KRBTEST.COM'} at 127.7.37.195:60473
I20260926 14:59:38.999033 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 14:59:39.005429 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.000399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39694 (local address 127.7.37.254:33149)
0926 14:59:39.000433 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:39.000434 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:39.000579 (+ 145us) server_negotiation.cc:407] Connection header received
0926 14:59:39.000631 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:39.000632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:39.000642 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:39.000694 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:39.001196 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.001634 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.002303 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.002468 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.002508 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.003475 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:39.003476 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:39.003476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:39.003480 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:39.004550 (+ 1070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:39.004852 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:39.004853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:39.004853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:39.004897 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:39.005049 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:39.005050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:39.005050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:39.005165 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:39.005236 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:39.005316 (+ 80us) server_negotiation.cc:299] Negotiation successful
0926 14:59:39.005375 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 14:59:39.006747 26430 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:59:39.008265 26430 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:59:39.013980 26897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.010015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:42993 (local address 127.0.0.1:46720)
0926 14:59:39.010316 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:39.010356 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:39.010416 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:39.010598 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:39.010601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:39.010603 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:39.010663 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.010664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.011937 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.011938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.013079 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.013081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.013720 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.013721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.013790 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.013802 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 14:59:39.013861 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 14:59:39.013909 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":34,"thread_start_us":54,"threads_started":1}
I20260926 14:59:39.014001 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.010134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46720 (local address 127.7.37.194:42993)
0926 14:59:39.010407 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:39.010411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:39.010422 (+ 11us) server_negotiation.cc:407] Connection header received
0926 14:59:39.010500 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:39.010502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:39.010517 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:39.010590 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:39.010743 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.011832 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.013164 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.013649 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.013710 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.013766 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:39.013879 (+ 113us) server_negotiation.cc:299] Negotiation successful
0926 14:59:39.013920 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":105,"threads_started":1}
I20260926 14:59:39.014245 26901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.010203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:41509 (local address 127.0.0.1:42650)
0926 14:59:39.010334 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:39.010356 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:39.010416 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:39.010717 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:39.010717 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:39.010719 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:39.010762 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.010764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.012207 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.012208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.013174 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.013177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.014042 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.014043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.014102 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.014105 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:59:39.014148 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 14:59:39.014192 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":55,"threads_started":1}
I20260926 14:59:39.014503 26902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.010253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42650 (local address 127.7.37.195:41509)
0926 14:59:39.010549 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:39.010552 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:39.010560 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:59:39.010588 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:39.010589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:39.010601 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:39.010663 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:39.011111 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.012113 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.013240 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.013908 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.013962 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.014009 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:39.014392 (+ 383us) server_negotiation.cc:299] Negotiation successful
0926 14:59:39.014435 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":62,"threads_started":1}
I20260926 14:59:39.014662 26898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.010054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:39505 (local address 127.0.0.1:35576)
0926 14:59:39.010319 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:39.010356 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:39.010418 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:39.010713 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:39.010714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:39.010716 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:39.010762 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.010764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.012583 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.012584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.013604 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.013607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.014457 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.014458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.014517 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.014521 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:59:39.014548 (+ 27us) client_negotiation.cc:240] Negotiation successful
0926 14:59:39.014587 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":9,"thread_start_us":43,"threads_started":1}
I20260926 14:59:39.014803 26689 tablet_service.cc:1467] Processing CreateTablet for tablet 1e0176ad20ec422d9c0b014a18be0260 (DEFAULT_TABLE table=test-table [id=cc3481ed2efd4dbca1b8d3c3e18eca6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:59:39.015012 26900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.010166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35576 (local address 127.7.37.193:39505)
0926 14:59:39.010535 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:39.010539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:39.010549 (+ 10us) server_negotiation.cc:407] Connection header received
0926 14:59:39.010584 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:39.010586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:39.010600 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:39.010709 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:39.010817 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.012478 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.013681 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.014363 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.014812 (+ 449us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.014876 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:39.014900 (+ 24us) server_negotiation.cc:299] Negotiation successful
0926 14:59:39.014942 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":79,"threads_started":1}
I20260926 14:59:39.015015 26823 tablet_service.cc:1467] Processing CreateTablet for tablet 1e0176ad20ec422d9c0b014a18be0260 (DEFAULT_TABLE table=test-table [id=cc3481ed2efd4dbca1b8d3c3e18eca6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:59:39.015167 26689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e0176ad20ec422d9c0b014a18be0260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:39.015278 26823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e0176ad20ec422d9c0b014a18be0260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:39.017421 26903 tablet_bootstrap.cc:492] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d: Bootstrap starting.
I20260926 14:59:39.017524 26904 tablet_bootstrap.cc:492] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7: Bootstrap starting.
I20260926 14:59:39.017830 26555 tablet_service.cc:1467] Processing CreateTablet for tablet 1e0176ad20ec422d9c0b014a18be0260 (DEFAULT_TABLE table=test-table [id=cc3481ed2efd4dbca1b8d3c3e18eca6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:59:39.018013 26903 tablet_bootstrap.cc:654] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:39.018043 26904 tablet_bootstrap.cc:654] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:39.018137 26555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e0176ad20ec422d9c0b014a18be0260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:39.018517 26904 log.cc:824] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:39.018522 26903 log.cc:824] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:39.019129 26904 tablet_bootstrap.cc:492] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7: No bootstrap required, opened a new log
I20260926 14:59:39.019192 26904 ts_tablet_manager.cc:1397] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:59:39.020581 26904 raft_consensus.cc:348] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee44fd027ec545899b90f629be6ff81d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42993 } } peers { permanent_uuid: "5681eb6fd2374a95a6bbf0f47fbd98b7" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41509 } } peers { permanent_uuid: "d8059e855aeb42e295c46af2988f3bab" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 39505 } }
I20260926 14:59:39.020709 26904 raft_consensus.cc:374] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:39.020740 26904 raft_consensus.cc:729] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5681eb6fd2374a95a6bbf0f47fbd98b7, State: Initialized, Role: FOLLOWER
I20260926 14:59:39.020845 26904 consensus_queue.cc:260] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee44fd027ec545899b90f629be6ff81d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42993 } } peers { permanent_uuid: "5681eb6fd2374a95a6bbf0f47fbd98b7" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41509 } } peers { permanent_uuid: "d8059e855aeb42e295c46af2988f3bab" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 39505 } }
I20260926 14:59:39.021029 26907 tablet_bootstrap.cc:492] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab: Bootstrap starting.
I20260926 14:59:39.021090 26904 ts_tablet_manager.cc:1428] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 14:59:39.021188 26889 heartbeater.cc:503] Master 127.7.37.254:33149 was elected leader, sending a full tablet report...
I20260926 14:59:39.021505 26903 tablet_bootstrap.cc:492] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d: No bootstrap required, opened a new log
I20260926 14:59:39.021574 26903 ts_tablet_manager.cc:1397] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260926 14:59:39.021586 26907 tablet_bootstrap.cc:654] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:39.022063 26907 log.cc:824] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:39.022756 26907 tablet_bootstrap.cc:492] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab: No bootstrap required, opened a new log
I20260926 14:59:39.022817 26907 ts_tablet_manager.cc:1397] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:59:39.023428 26903 raft_consensus.cc:348] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee44fd027ec545899b90f629be6ff81d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42993 } } peers { permanent_uuid: "5681eb6fd2374a95a6bbf0f47fbd98b7" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41509 } } peers { permanent_uuid: "d8059e855aeb42e295c46af2988f3bab" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 39505 } }
I20260926 14:59:39.023597 26903 raft_consensus.cc:374] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:39.023630 26903 raft_consensus.cc:729] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee44fd027ec545899b90f629be6ff81d, State: Initialized, Role: FOLLOWER
I20260926 14:59:39.023772 26903 consensus_queue.cc:260] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee44fd027ec545899b90f629be6ff81d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42993 } } peers { permanent_uuid: "5681eb6fd2374a95a6bbf0f47fbd98b7" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41509 } } peers { permanent_uuid: "d8059e855aeb42e295c46af2988f3bab" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 39505 } }
I20260926 14:59:39.024025 26903 ts_tablet_manager.cc:1428] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 14:59:39.024036 26755 heartbeater.cc:503] Master 127.7.37.254:33149 was elected leader, sending a full tablet report...
I20260926 14:59:39.024590 26907 raft_consensus.cc:348] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee44fd027ec545899b90f629be6ff81d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42993 } } peers { permanent_uuid: "5681eb6fd2374a95a6bbf0f47fbd98b7" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41509 } } peers { permanent_uuid: "d8059e855aeb42e295c46af2988f3bab" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 39505 } }
I20260926 14:59:39.024752 26907 raft_consensus.cc:374] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:39.024794 26907 raft_consensus.cc:729] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8059e855aeb42e295c46af2988f3bab, State: Initialized, Role: FOLLOWER
I20260926 14:59:39.024889 26907 consensus_queue.cc:260] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee44fd027ec545899b90f629be6ff81d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42993 } } peers { permanent_uuid: "5681eb6fd2374a95a6bbf0f47fbd98b7" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41509 } } peers { permanent_uuid: "d8059e855aeb42e295c46af2988f3bab" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 39505 } }
I20260926 14:59:39.025115 26907 ts_tablet_manager.cc:1428] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 14:59:39.025146 26621 heartbeater.cc:503] Master 127.7.37.254:33149 was elected leader, sending a full tablet report...
I20260926 14:59:39.038136 26911 raft_consensus.cc:482] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:59:39.038203 26911 raft_consensus.cc:504] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee44fd027ec545899b90f629be6ff81d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42993 } } peers { permanent_uuid: "5681eb6fd2374a95a6bbf0f47fbd98b7" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41509 } } peers { permanent_uuid: "d8059e855aeb42e295c46af2988f3bab" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 39505 } }
I20260926 14:59:39.038452 26911 leader_election.cc:288] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee44fd027ec545899b90f629be6ff81d (127.7.37.194:42993), 5681eb6fd2374a95a6bbf0f47fbd98b7 (127.7.37.195:41509)
I20260926 14:59:39.041020 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.038629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:36781 (local address 127.7.37.194:42993)
0926 14:59:39.038668 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:39.038670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:39.038678 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:59:39.038697 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:39.038698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:39.038705 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:39.038797 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:39.038930 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.039401 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.040343 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.040734 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.040783 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.040788 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:39.040931 (+ 143us) server_negotiation.cc:299] Negotiation successful
0926 14:59:39.040961 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 14:59:39.041013 26623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.038568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:42993 (local address 127.7.37.193:36781)
0926 14:59:39.038610 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:39.038620 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:39.038654 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:39.038815 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:39.038816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:39.038818 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:39.038868 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.038870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.039464 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.039465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.040271 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.040273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.040798 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.040799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.040869 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.040872 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:59:39.040918 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 14:59:39.040953 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260926 14:59:39.041082 26902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.038726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:34919 (local address 127.7.37.195:41509)
0926 14:59:39.038777 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:39.038779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:39.038859 (+ 80us) server_negotiation.cc:407] Connection header received
0926 14:59:39.038894 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:39.038895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:39.038903 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:39.038949 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:39.039073 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.039526 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.040452 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.040799 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.040863 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.040870 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:39.041001 (+ 131us) server_negotiation.cc:299] Negotiation successful
0926 14:59:39.041032 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 14:59:39.041303 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.038684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:41509 (local address 127.7.37.193:34919)
0926 14:59:39.038800 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:39.038810 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:39.038859 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:39.038950 (+ 91us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:39.038950 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:39.038952 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:39.039007 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.039008 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.039596 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.039596 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.040380 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.040382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.040873 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.040874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.040937 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.040941 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 14:59:39.040992 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 14:59:39.041024 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260926 14:59:39.041292 26709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0176ad20ec422d9c0b014a18be0260" candidate_uuid: "d8059e855aeb42e295c46af2988f3bab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee44fd027ec545899b90f629be6ff81d" is_pre_election: true
I20260926 14:59:39.041386 26709 raft_consensus.cc:2454] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8059e855aeb42e295c46af2988f3bab in term 0.
I20260926 14:59:39.041621 26843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0176ad20ec422d9c0b014a18be0260" candidate_uuid: "d8059e855aeb42e295c46af2988f3bab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5681eb6fd2374a95a6bbf0f47fbd98b7" is_pre_election: true
I20260926 14:59:39.041667 26508 leader_election.cc:302] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8059e855aeb42e295c46af2988f3bab, ee44fd027ec545899b90f629be6ff81d; no voters:
I20260926 14:59:39.041720 26843 raft_consensus.cc:2454] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8059e855aeb42e295c46af2988f3bab in term 0.
I20260926 14:59:39.041790 26911 raft_consensus.cc:2781] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:59:39.041837 26911 raft_consensus.cc:482] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:59:39.041862 26911 raft_consensus.cc:3037] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:39.042383 26911 raft_consensus.cc:504] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee44fd027ec545899b90f629be6ff81d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42993 } } peers { permanent_uuid: "5681eb6fd2374a95a6bbf0f47fbd98b7" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41509 } } peers { permanent_uuid: "d8059e855aeb42e295c46af2988f3bab" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 39505 } }
I20260926 14:59:39.042469 26911 leader_election.cc:288] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab [CANDIDATE]: Term 1 election: Requested vote from peers ee44fd027ec545899b90f629be6ff81d (127.7.37.194:42993), 5681eb6fd2374a95a6bbf0f47fbd98b7 (127.7.37.195:41509)
I20260926 14:59:39.042637 26709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0176ad20ec422d9c0b014a18be0260" candidate_uuid: "d8059e855aeb42e295c46af2988f3bab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee44fd027ec545899b90f629be6ff81d"
I20260926 14:59:39.042637 26843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0176ad20ec422d9c0b014a18be0260" candidate_uuid: "d8059e855aeb42e295c46af2988f3bab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5681eb6fd2374a95a6bbf0f47fbd98b7"
I20260926 14:59:39.042704 26843 raft_consensus.cc:3037] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:39.042704 26709 raft_consensus.cc:3037] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:39.043416 26843 raft_consensus.cc:2454] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8059e855aeb42e295c46af2988f3bab in term 1.
I20260926 14:59:39.043416 26709 raft_consensus.cc:2454] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8059e855aeb42e295c46af2988f3bab in term 1.
I20260926 14:59:39.043560 26508 leader_election.cc:302] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8059e855aeb42e295c46af2988f3bab, ee44fd027ec545899b90f629be6ff81d; no voters:
I20260926 14:59:39.043635 26911 raft_consensus.cc:2781] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:39.043777 26911 raft_consensus.cc:686] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab [term 1 LEADER]: Becoming Leader. State: Replica: d8059e855aeb42e295c46af2988f3bab, State: Running, Role: LEADER
I20260926 14:59:39.043853 26911 consensus_queue.cc:237] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee44fd027ec545899b90f629be6ff81d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42993 } } peers { permanent_uuid: "5681eb6fd2374a95a6bbf0f47fbd98b7" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41509 } } peers { permanent_uuid: "d8059e855aeb42e295c46af2988f3bab" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 39505 } }
I20260926 14:59:39.044485 26430 catalog_manager.cc:5495] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab reported cstate change: term changed from 0 to 1, leader changed from <none> to d8059e855aeb42e295c46af2988f3bab (127.7.37.193). New cstate: current_term: 1 leader_uuid: "d8059e855aeb42e295c46af2988f3bab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee44fd027ec545899b90f629be6ff81d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5681eb6fd2374a95a6bbf0f47fbd98b7" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8059e855aeb42e295c46af2988f3bab" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 39505 } health_report { overall_health: HEALTHY } } }
I20260926 14:59:39.054697 26900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.051756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35590 (local address 127.7.37.193:39505)
0926 14:59:39.051876 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:39.051877 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:39.051889 (+ 12us) server_negotiation.cc:407] Connection header received
0926 14:59:39.051919 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:39.051920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:39.051939 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:39.052020 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:59:39.052169 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.052861 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.053813 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.054132 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.054203 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.054403 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:59:39.054495 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:59:39.054574 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:39.054599 (+ 25us) server_negotiation.cc:299] Negotiation successful
0926 14:59:39.054617 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 14:59:39.056166 26709 raft_consensus.cc:1261] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d [term 1 FOLLOWER]: Refusing update from remote peer d8059e855aeb42e295c46af2988f3bab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:59:39.056167 26843 raft_consensus.cc:1261] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7 [term 1 FOLLOWER]: Refusing update from remote peer d8059e855aeb42e295c46af2988f3bab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:59:39.056473 26913 consensus_queue.cc:1035] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee44fd027ec545899b90f629be6ff81d" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:59:39.056603 26911 consensus_queue.cc:1035] T 1e0176ad20ec422d9c0b014a18be0260 P d8059e855aeb42e295c46af2988f3bab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5681eb6fd2374a95a6bbf0f47fbd98b7" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 41509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:59:39.058610 26916 mvcc.cc:205] Tried to move back new op lower bound from 7333620855011647488 to 7333620854963994624. Current Snapshot: MvccSnapshot[applied={T|T < 7333620855011647488}]
I20260926 14:59:39.059405 26918 mvcc.cc:205] Tried to move back new op lower bound from 7333620855011647488 to 7333620854963994624. Current Snapshot: MvccSnapshot[applied={T|T < 7333620855011647488}]
I20260926 14:59:39.060760 26430 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:59:39.060827 26430 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:59:39.061358 26430 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72b32861e1be4283b806d9057aba77ca: Sending DeleteTablet for 3 replicas of tablet 1e0176ad20ec422d9c0b014a18be0260
I20260926 14:59:39.061612 26689 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e0176ad20ec422d9c0b014a18be0260 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:59:39 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:46720
I20260926 14:59:39.061654 26823 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e0176ad20ec422d9c0b014a18be0260 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:59:39 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:42650
I20260926 14:59:39.061695 26555 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e0176ad20ec422d9c0b014a18be0260 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:59:39 UTC) from {username='kudu', principal='kudu/127.7.37.254@KRBTEST.COM'} at 127.0.0.1:35576
I20260926 14:59:39.061869 26925 tablet_replica.cc:330] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d: stopping tablet replica
I20260926 14:59:39.061959 26925 raft_consensus.cc:2229] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:59:39.061954 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 26492
I20260926 14:59:39.062463 26926 tablet_replica.cc:330] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7: stopping tablet replica
I20260926 14:59:39.062547 26926 raft_consensus.cc:2229] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:59:39.062623 26926 raft_consensus.cc:2258] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:59:39.062750 26925 raft_consensus.cc:2258] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:59:39.063145 26926 ts_tablet_manager.cc:1905] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:59:39.064451 26925 ts_tablet_manager.cc:1905] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:59:39.064618 26926 ts_tablet_manager.cc:1918] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:59:39.064682 26926 log.cc:1196] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/1e0176ad20ec422d9c0b014a18be0260
I20260926 14:59:39.064914 26926 ts_tablet_manager.cc:1939] T 1e0176ad20ec422d9c0b014a18be0260 P 5681eb6fd2374a95a6bbf0f47fbd98b7: Deleting consensus metadata
I20260926 14:59:39.065268 26414 catalog_manager.cc:4841] TS 5681eb6fd2374a95a6bbf0f47fbd98b7 (127.7.37.195:41509): tablet 1e0176ad20ec422d9c0b014a18be0260 (table test-table [id=cc3481ed2efd4dbca1b8d3c3e18eca6a]) successfully deleted
I20260926 14:59:39.065732 26925 ts_tablet_manager.cc:1918] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:59:39.065781 26925 log.cc:1196] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/1e0176ad20ec422d9c0b014a18be0260
I20260926 14:59:39.066082 26925 ts_tablet_manager.cc:1939] T 1e0176ad20ec422d9c0b014a18be0260 P ee44fd027ec545899b90f629be6ff81d: Deleting consensus metadata
W20260926 14:59:39.066437 26415 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.37.193:39505 (error 108)
I20260926 14:59:39.066540 26414 catalog_manager.cc:4841] TS ee44fd027ec545899b90f629be6ff81d (127.7.37.194:42993): tablet 1e0176ad20ec422d9c0b014a18be0260 (table test-table [id=cc3481ed2efd4dbca1b8d3c3e18eca6a]) successfully deleted
I20260926 14:59:39.066684 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 26626
I20260926 14:59:39.066812 26898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.066695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:39505 (local address 127.0.0.1:35598)
0926 14:59:39.066745 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:39.066780 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.193:39505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260926 14:59:39.070503 26415 catalog_manager.cc:4584] TS d8059e855aeb42e295c46af2988f3bab (127.7.37.193:39505): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e0176ad20ec422d9c0b014a18be0260: Network error: Client connection negotiation failed: client connection to 127.7.37.193:39505: connect: Connection refused (error 111)
I20260926 14:59:39.070672 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 26760
I20260926 14:59:39.074812 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 26398
2026-09-26T14:59:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1720 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-26T14:59:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:39Z Disabled control of system clock
I20260926 14:59:39.099017 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36523
--webserver_interface=127.7.37.254
--webserver_port=0
--builtin_ntp_servers=127.7.37.212:44619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:36523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 14:59:39.174793 26932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:39.174974 26932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:39.174993 26932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:39.176057 26932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:39.176091 26932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:39.176102 26932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:39.176113 26932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:39.176124 26932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:39.177366 26932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:44619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:36523
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:39.177533 26932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:39.177752 26932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:39.182080 26938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:39.182226 26932 server_base.cc:1034] running on GCE node
W20260926 14:59:39.182054 26940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:39.182065 26937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:39.182521 26932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:39.182797 26932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:39.183933 26932 hybrid_clock.cc:648] HybridClock initialized: now 1790434779183912 us; error 38 us; skew 500 ppm
I20260926 14:59:39.190217 26932 webserver.cc:460] Webserver started at http://127.7.37.254:41973/ using document root <none> and password file <none>
I20260926 14:59:39.190428 26932 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:39.190477 26932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:39.190629 26932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:39.191561 26932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "f85049d6f889448e84ba8b941a893d73"
format_stamp: "Formatted at 2026-09-26 14:59:39 on dist-test-slave-f9xb"
server_key: "70ee4ed12df82dd2e2d4345179035d73"
server_key_iv: "ff5e29341325dd0541ed6ddae39cc2ab"
server_key_version: "encryptionkey@0"
I20260926 14:59:39.191922 26932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "f85049d6f889448e84ba8b941a893d73"
format_stamp: "Formatted at 2026-09-26 14:59:39 on dist-test-slave-f9xb"
server_key: "70ee4ed12df82dd2e2d4345179035d73"
server_key_iv: "ff5e29341325dd0541ed6ddae39cc2ab"
server_key_version: "encryptionkey@0"
I20260926 14:59:39.193156 26932 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 14:59:39.193778 26946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:39.193929 26932 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:59:39.194005 26932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "f85049d6f889448e84ba8b941a893d73"
format_stamp: "Formatted at 2026-09-26 14:59:39 on dist-test-slave-f9xb"
server_key: "70ee4ed12df82dd2e2d4345179035d73"
server_key_iv: "ff5e29341325dd0541ed6ddae39cc2ab"
server_key_version: "encryptionkey@0"
I20260926 14:59:39.194060 26932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:39.205802 26932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:39.206024 26932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:39.206133 26932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:39.209573 26932 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:36523
I20260926 14:59:39.209610 26998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:36523 every 8 connection(s)
I20260926 14:59:39.209885 26932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:39.210686 26999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:39.212548 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 26932
I20260926 14:59:39.212644 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:39.212759 26999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73: Bootstrap starting.
I20260926 14:59:39.212847 7319 external_mini_cluster.cc:1428] Setting key 5ac464fb07d207f8c8fe1e7b53297759
I20260926 14:59:39.213338 26999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:39.213636 26999 log.cc:824] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:39.214298 26999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73: No bootstrap required, opened a new log
I20260926 14:59:39.216374 26999 raft_consensus.cc:348] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f85049d6f889448e84ba8b941a893d73" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36523 } }
I20260926 14:59:39.216522 26999 raft_consensus.cc:374] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:39.216562 26999 raft_consensus.cc:729] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f85049d6f889448e84ba8b941a893d73, State: Initialized, Role: FOLLOWER
I20260926 14:59:39.216672 26999 consensus_queue.cc:260] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f85049d6f889448e84ba8b941a893d73" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36523 } }
I20260926 14:59:39.216746 26999 raft_consensus.cc:388] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:39.216779 26999 raft_consensus.cc:482] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:39.216820 26999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:39.217386 26999 raft_consensus.cc:504] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f85049d6f889448e84ba8b941a893d73" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36523 } }
I20260926 14:59:39.217465 26999 leader_election.cc:302] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f85049d6f889448e84ba8b941a893d73; no voters:
I20260926 14:59:39.217608 26999 leader_election.cc:288] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:39.217635 27004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:39.217731 27004 raft_consensus.cc:686] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73 [term 1 LEADER]: Becoming Leader. State: Replica: f85049d6f889448e84ba8b941a893d73, State: Running, Role: LEADER
I20260926 14:59:39.217809 27004 consensus_queue.cc:237] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f85049d6f889448e84ba8b941a893d73" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36523 } }
I20260926 14:59:39.217871 26999 sys_catalog.cc:564] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:39.218098 27005 sys_catalog.cc:455] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f85049d6f889448e84ba8b941a893d73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f85049d6f889448e84ba8b941a893d73" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36523 } } }
I20260926 14:59:39.218209 27005 sys_catalog.cc:458] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:39.218364 27006 sys_catalog.cc:455] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f85049d6f889448e84ba8b941a893d73. Latest consensus state: current_term: 1 leader_uuid: "f85049d6f889448e84ba8b941a893d73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f85049d6f889448e84ba8b941a893d73" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36523 } } }
I20260926 14:59:39.218422 27006 sys_catalog.cc:458] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:39.218436 27002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.213258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38390 (local address 127.7.37.254:36523)
0926 14:59:39.213449 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:39.213457 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:39.213491 (+ 34us) server_negotiation.cc:407] Connection header received
0926 14:59:39.213941 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:39.213947 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:39.214000 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:39.214129 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:39.214333 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.215672 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.216416 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.216984 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.217105 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.217623 (+ 518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:39.217626 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:39.217626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:39.217811 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:39.217852 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:39.217866 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:39.217921 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:39.218167 (+ 246us) server_negotiation.cc:299] Negotiation successful
0926 14:59:39.218253 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260926 14:59:39.218709 27010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:39.219108 27010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:39.224553 27010 catalog_manager.cc:1347] Generated new cluster ID: 42c9b60e5e074282bf0f5cf9c06287eb
I20260926 14:59:39.224602 27010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:59:39.253573 27010 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:39.254011 27010 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:39.257488 27010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73: Generated new TSK 0
I20260926 14:59:39.257611 27010 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:39.262789 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.193:0
--local_ip_for_outbound_sockets=127.7.37.193
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36523
--builtin_ntp_servers=127.7.37.212:44619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 14:59:39.345170 27023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:39.345355 27023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:39.345377 27023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:39.346422 27023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:39.346459 27023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:39.346484 27023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.193
I20260926 14:59:39.347764 27023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:44619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.37.193
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.193
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:39.347952 27023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:39.348184 27023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:39.352631 27031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:39.352738 27028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:39.352632 27029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:39.352877 27023 server_base.cc:1034] running on GCE node
I20260926 14:59:39.353060 27023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:39.353288 27023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:39.354420 27023 hybrid_clock.cc:648] HybridClock initialized: now 1790434779354404 us; error 30 us; skew 500 ppm
I20260926 14:59:39.355464 27023 webserver.cc:460] Webserver started at http://127.7.37.193:38635/ using document root <none> and password file <none>
I20260926 14:59:39.355670 27023 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:39.355717 27023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:39.355854 27023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:39.356798 27023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-0/data/instance:
uuid: "90046ca8651f4494967f7ad3d985ac34"
format_stamp: "Formatted at 2026-09-26 14:59:39 on dist-test-slave-f9xb"
server_key: "b52052a851e5b8df542520718911fc83"
server_key_iv: "80af1f9c6c40b3b1c762bbbc0b815691"
server_key_version: "encryptionkey@0"
I20260926 14:59:39.357102 27023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance:
uuid: "90046ca8651f4494967f7ad3d985ac34"
format_stamp: "Formatted at 2026-09-26 14:59:39 on dist-test-slave-f9xb"
server_key: "b52052a851e5b8df542520718911fc83"
server_key_iv: "80af1f9c6c40b3b1c762bbbc0b815691"
server_key_version: "encryptionkey@0"
I20260926 14:59:39.358229 27023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:39.358814 27037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:39.358954 27023 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:39.359027 27023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-0/wal
uuid: "90046ca8651f4494967f7ad3d985ac34"
format_stamp: "Formatted at 2026-09-26 14:59:39 on dist-test-slave-f9xb"
server_key: "b52052a851e5b8df542520718911fc83"
server_key_iv: "80af1f9c6c40b3b1c762bbbc0b815691"
server_key_version: "encryptionkey@0"
I20260926 14:59:39.359088 27023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:39.370560 27023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:39.370781 27023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:39.370888 27023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:39.371064 27023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:39.371332 27023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:59:39.371363 27023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:39.371387 27023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:59:39.371412 27023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:39.376539 27023 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.193:45193
I20260926 14:59:39.376596 27150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.193:45193 every 8 connection(s)
I20260926 14:59:39.376829 27023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-0/data/info.pb
I20260926 14:59:39.377319 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 27023
I20260926 14:59:39.377419 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-0/wal/instance
I20260926 14:59:39.377633 7319 external_mini_cluster.cc:1428] Setting key 9f0a78827bcf92f57e0f0a5ba33bd6a9
I20260926 14:59:39.378544 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.194:0
--local_ip_for_outbound_sockets=127.7.37.194
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36523
--builtin_ntp_servers=127.7.37.212:44619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 14:59:39.382230 27002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.377402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:58275 (local address 127.7.37.254:36523)
0926 14:59:39.377444 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:39.377446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:39.377709 (+ 263us) server_negotiation.cc:407] Connection header received
0926 14:59:39.378213 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:39.378214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:39.378225 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:39.378278 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:39.379073 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.379768 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.380983 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.381252 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.381320 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.381993 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:39.381994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:39.381995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:39.381999 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:39.382016 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:39.382021 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:39.382083 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:39.382129 (+ 46us) server_negotiation.cc:299] Negotiation successful
0926 14:59:39.382154 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 14:59:39.382360 27153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.377401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36523 (local address 127.7.37.193:58275)
0926 14:59:39.377582 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:39.377621 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:39.378112 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:39.378339 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:39.378343 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:39.378353 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:39.378983 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.378988 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.379854 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.379855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.380901 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.380904 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.381325 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.381326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.381426 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.381870 (+ 444us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:59:39.381871 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:39.381896 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:59:39.381915 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:39.382083 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:39.382085 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:39.382087 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:59:39.382127 (+ 40us) client_negotiation.cc:240] Negotiation successful
0926 14:59:39.382199 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260926 14:59:39.382771 27151 heartbeater.cc:347] Connected to a master server at 127.7.37.254:36523
I20260926 14:59:39.382843 27151 heartbeater.cc:464] Registering TS with master...
I20260926 14:59:39.382990 27151 heartbeater.cc:511] Master 127.7.37.254:36523 requested a full tablet report, sending...
I20260926 14:59:39.383399 26963 ts_manager.cc:194] Registered new tserver with Master: 90046ca8651f4494967f7ad3d985ac34 (127.7.37.193:45193)
I20260926 14:59:39.384032 26963 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.37.193:58275
W20260926 14:59:39.454687 27154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:39.454849 27154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:39.454869 27154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:39.455982 27154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:39.456018 27154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:39.456045 27154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.194
I20260926 14:59:39.457331 27154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:44619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.37.194
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.194
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:39.457520 27154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:39.457758 27154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:39.462157 27159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:39.462167 27162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:39.462347 27160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:39.462378 27154 server_base.cc:1034] running on GCE node
I20260926 14:59:39.462564 27154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:39.462790 27154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:39.463910 27154 hybrid_clock.cc:648] HybridClock initialized: now 1790434779463902 us; error 24 us; skew 500 ppm
I20260926 14:59:39.464936 27154 webserver.cc:460] Webserver started at http://127.7.37.194:37245/ using document root <none> and password file <none>
I20260926 14:59:39.465137 27154 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:39.465183 27154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:39.465296 27154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:39.466199 27154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-1/data/instance:
uuid: "f7211846b090478cadeefd1a491e57b0"
format_stamp: "Formatted at 2026-09-26 14:59:39 on dist-test-slave-f9xb"
server_key: "eb12ad3b6e6f93b914692be40d9db7f0"
server_key_iv: "49440632b2c76baed5a168bc0d9b5de0"
server_key_version: "encryptionkey@0"
I20260926 14:59:39.466491 27154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance:
uuid: "f7211846b090478cadeefd1a491e57b0"
format_stamp: "Formatted at 2026-09-26 14:59:39 on dist-test-slave-f9xb"
server_key: "eb12ad3b6e6f93b914692be40d9db7f0"
server_key_iv: "49440632b2c76baed5a168bc0d9b5de0"
server_key_version: "encryptionkey@0"
I20260926 14:59:39.467590 27154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 14:59:39.468243 27168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:39.468384 27154 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:39.468457 27154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-1/wal
uuid: "f7211846b090478cadeefd1a491e57b0"
format_stamp: "Formatted at 2026-09-26 14:59:39 on dist-test-slave-f9xb"
server_key: "eb12ad3b6e6f93b914692be40d9db7f0"
server_key_iv: "49440632b2c76baed5a168bc0d9b5de0"
server_key_version: "encryptionkey@0"
I20260926 14:59:39.468521 27154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:39.499629 27154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:39.499889 27154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:39.500008 27154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:39.500191 27154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:39.500456 27154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:59:39.500487 27154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:39.500514 27154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:59:39.500533 27154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:39.505517 27154 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.194:42401
I20260926 14:59:39.505548 27281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.194:42401 every 8 connection(s)
I20260926 14:59:39.505873 27154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-1/data/info.pb
I20260926 14:59:39.510104 27002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.506475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.194:58983 (local address 127.7.37.254:36523)
0926 14:59:39.506531 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:39.506533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:39.506710 (+ 177us) server_negotiation.cc:407] Connection header received
0926 14:59:39.507113 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:39.507114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:39.507122 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:39.507175 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:39.507736 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.508211 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.509083 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.509272 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.509322 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.509922 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:39.509923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:39.509923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:39.509927 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:39.509940 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:39.509943 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:39.509979 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:39.510031 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 14:59:39.510050 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 14:59:39.510273 27284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.506451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36523 (local address 127.7.37.194:58983)
0926 14:59:39.506605 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:39.506640 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:39.507046 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:39.507226 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:39.507229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:39.507237 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:39.507671 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.507675 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.508270 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.508271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.509014 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.509016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.509329 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.509330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.509423 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.509797 (+ 374us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:59:39.509797 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:39.509835 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:59:39.509855 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:39.509980 (+ 125us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:39.509982 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:39.509984 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:59:39.510025 (+ 41us) client_negotiation.cc:240] Negotiation successful
0926 14:59:39.510093 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":70,"threads_started":1}
I20260926 14:59:39.510686 27282 heartbeater.cc:347] Connected to a master server at 127.7.37.254:36523
I20260926 14:59:39.510761 27282 heartbeater.cc:464] Registering TS with master...
I20260926 14:59:39.510921 27282 heartbeater.cc:511] Master 127.7.37.254:36523 requested a full tablet report, sending...
I20260926 14:59:39.511225 26963 ts_manager.cc:194] Registered new tserver with Master: f7211846b090478cadeefd1a491e57b0 (127.7.37.194:42401)
I20260926 14:59:39.511554 26963 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.37.194:58983
I20260926 14:59:39.513106 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 27154
I20260926 14:59:39.513206 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-1/wal/instance
I20260926 14:59:39.513341 7319 external_mini_cluster.cc:1428] Setting key c13887114445b9933e4301ce27b79dda
I20260926 14:59:39.514084 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.37.195:0
--local_ip_for_outbound_sockets=127.7.37.195
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36523
--builtin_ntp_servers=127.7.37.212:44619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 14:59:39.589094 27285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:39.589267 27285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:39.589287 27285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:39.590351 27285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:39.590384 27285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:39.590407 27285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.37.195
I20260926 14:59:39.591640 27285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:44619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.37.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.37.195
--webserver_port=0
--tserver_master_addrs=127.7.37.254:36523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.37.195
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:39.591827 27285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:39.592056 27285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:39.596339 27290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:39.596349 27291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:39.596381 27293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:39.596501 27285 server_base.cc:1034] running on GCE node
I20260926 14:59:39.596709 27285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:39.596930 27285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:39.598059 27285 hybrid_clock.cc:648] HybridClock initialized: now 1790434779598045 us; error 28 us; skew 500 ppm
I20260926 14:59:39.599053 27285 webserver.cc:460] Webserver started at http://127.7.37.195:43133/ using document root <none> and password file <none>
I20260926 14:59:39.599267 27285 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:39.599314 27285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:39.599429 27285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:39.600369 27285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-2/data/instance:
uuid: "d9032abd3e5b42fb8ccad918ab3abfdb"
format_stamp: "Formatted at 2026-09-26 14:59:39 on dist-test-slave-f9xb"
server_key: "2d646af04e5632fb63221aaa74ca7581"
server_key_iv: "1640d6a76ffb74e56d1c8f9e75b167f7"
server_key_version: "encryptionkey@0"
I20260926 14:59:39.600664 27285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance:
uuid: "d9032abd3e5b42fb8ccad918ab3abfdb"
format_stamp: "Formatted at 2026-09-26 14:59:39 on dist-test-slave-f9xb"
server_key: "2d646af04e5632fb63221aaa74ca7581"
server_key_iv: "1640d6a76ffb74e56d1c8f9e75b167f7"
server_key_version: "encryptionkey@0"
I20260926 14:59:39.601747 27285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:39.602353 27299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:39.602489 27285 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:39.602560 27285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-2/wal
uuid: "d9032abd3e5b42fb8ccad918ab3abfdb"
format_stamp: "Formatted at 2026-09-26 14:59:39 on dist-test-slave-f9xb"
server_key: "2d646af04e5632fb63221aaa74ca7581"
server_key_iv: "1640d6a76ffb74e56d1c8f9e75b167f7"
server_key_version: "encryptionkey@0"
I20260926 14:59:39.602620 27285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:39.613556 27285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:39.613760 27285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:39.613865 27285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:39.614033 27285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 14:59:39.614284 27285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 14:59:39.614315 27285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:39.614338 27285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 14:59:39.614363 27285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:39.619541 27285 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.195:35515
I20260926 14:59:39.619598 27412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.195:35515 every 8 connection(s)
I20260926 14:59:39.619872 27285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-2/data/info.pb
I20260926 14:59:39.624107 27002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.620425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.195:42169 (local address 127.7.37.254:36523)
0926 14:59:39.620481 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:39.620483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:39.620661 (+ 178us) server_negotiation.cc:407] Connection header received
0926 14:59:39.621023 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:39.621024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:39.621033 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:39.621078 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:39.621603 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.622065 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.622949 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.623129 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.623190 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.623856 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:39.623858 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:39.623858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:39.623862 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:39.623875 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:39.623879 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:39.623916 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:39.624009 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 14:59:39.624030 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 14:59:39.624258 27415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.620410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36523 (local address 127.7.37.195:42169)
0926 14:59:39.620559 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:39.620585 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:39.620956 (+ 371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:39.621121 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:39.621123 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:39.621130 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 14:59:39.621549 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.621553 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.622137 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.622139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.622884 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.622886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.623206 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.623206 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.623299 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.623675 (+ 376us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 14:59:39.623675 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 14:59:39.623698 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 14:59:39.623718 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 14:59:39.623926 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 14:59:39.623931 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 14:59:39.623933 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:59:39.623997 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 14:59:39.624073 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":79,"threads_started":1}
I20260926 14:59:39.624709 27413 heartbeater.cc:347] Connected to a master server at 127.7.37.254:36523
I20260926 14:59:39.624778 27413 heartbeater.cc:464] Registering TS with master...
I20260926 14:59:39.624894 27413 heartbeater.cc:511] Master 127.7.37.254:36523 requested a full tablet report, sending...
I20260926 14:59:39.625172 26963 ts_manager.cc:194] Registered new tserver with Master: d9032abd3e5b42fb8ccad918ab3abfdb (127.7.37.195:35515)
I20260926 14:59:39.625506 26963 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.37.195:42169
I20260926 14:59:39.627842 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 27285
I20260926 14:59:39.627931 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-2/wal/instance
I20260926 14:59:39.628065 7319 external_mini_cluster.cc:1428] Setting key 074e40da647c18d1490830805ee05fab
I20260926 14:59:39.628736 7319 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 14:59:39.632105 27002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.629879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38398 (local address 127.7.37.254:36523)
0926 14:59:39.629931 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:39.629933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:39.630022 (+ 89us) server_negotiation.cc:407] Connection header received
0926 14:59:39.630057 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:39.630058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:39.630064 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:39.630104 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:39.630221 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.630640 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.631296 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.631456 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.631495 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.631945 (+ 450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:39.631946 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:39.631946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:39.631949 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:39.631960 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:39.631963 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:39.631998 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:39.632035 (+ 37us) server_negotiation.cc:299] Negotiation successful
0926 14:59:39.632050 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 14:59:39.633298 26963 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 14:59:39.634640 26963 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 14:59:39.640100 27422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.636369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:45193 (local address 127.0.0.1:49268)
0926 14:59:39.636633 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:39.636643 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:39.636693 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:39.637122 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:39.637123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:39.637125 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:39.637159 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.637160 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.638393 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.638394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.639135 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.639137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.639836 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.639837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.639911 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.639923 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 14:59:39.639982 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 14:59:39.640031 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":73,"threads_started":1}
I20260926 14:59:39.640190 27426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.636568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50098 (local address 127.7.37.195:35515)
0926 14:59:39.636690 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:39.636694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:39.636825 (+ 131us) server_negotiation.cc:407] Connection header received
0926 14:59:39.636861 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:39.636864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:39.636876 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:39.636940 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:39.637038 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.637830 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.639215 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.639916 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.639980 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.640053 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:39.640091 (+ 38us) server_negotiation.cc:299] Negotiation successful
0926 14:59:39.640123 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":62,"threads_started":1}
I20260926 14:59:39.640290 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.636335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:42401 (local address 127.0.0.1:44962)
0926 14:59:39.636588 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:39.636621 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:39.636691 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:39.636878 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:39.636880 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:39.636883 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:39.636937 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.636938 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.638149 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.638150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.639196 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.639197 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.640086 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.640087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.640145 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.640148 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:59:39.640189 (+ 41us) client_negotiation.cc:240] Negotiation successful
0926 14:59:39.640226 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":50,"threads_started":1}
I20260926 14:59:39.640442 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.636465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49268 (local address 127.7.37.193:45193)
0926 14:59:39.636990 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:39.636994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:39.637003 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:59:39.637041 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:39.637043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:39.637057 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:39.637289 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:39.637357 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.638326 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.639232 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.639771 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.639836 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.639927 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:39.640305 (+ 378us) server_negotiation.cc:299] Negotiation successful
0926 14:59:39.640359 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":72,"threads_started":1}
I20260926 14:59:39.640630 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.636428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44962 (local address 127.7.37.194:42401)
0926 14:59:39.636736 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:39.636741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:39.636750 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:59:39.636784 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:39.636786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:39.636800 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:39.636866 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:39.637013 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.638070 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.639275 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.640000 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.640389 (+ 389us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.640485 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:39.640511 (+ 26us) server_negotiation.cc:299] Negotiation successful
0926 14:59:39.640555 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":69,"threads_started":1}
I20260926 14:59:39.640699 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.636512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:35515 (local address 127.0.0.1:50098)
0926 14:59:39.636774 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:39.636780 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:39.636815 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:39.636948 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:39.636948 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:39.636950 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:39.636991 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.636993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.637906 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.637907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.639127 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.639129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.639991 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.639992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.640041 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.640044 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:59:39.640073 (+ 29us) client_negotiation.cc:240] Negotiation successful
0926 14:59:39.640102 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":16,"thread_start_us":59,"threads_started":1}
I20260926 14:59:39.641037 27085 tablet_service.cc:1467] Processing CreateTablet for tablet 39970ef3d54e44c6adb7a45ce093b5fd (DEFAULT_TABLE table=test-table [id=059fecb1190f4066af3b51bf6a07a6eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:59:39.641273 27085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39970ef3d54e44c6adb7a45ce093b5fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:39.641252 27347 tablet_service.cc:1467] Processing CreateTablet for tablet 39970ef3d54e44c6adb7a45ce093b5fd (DEFAULT_TABLE table=test-table [id=059fecb1190f4066af3b51bf6a07a6eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:59:39.641243 27216 tablet_service.cc:1467] Processing CreateTablet for tablet 39970ef3d54e44c6adb7a45ce093b5fd (DEFAULT_TABLE table=test-table [id=059fecb1190f4066af3b51bf6a07a6eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 14:59:39.641538 27347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39970ef3d54e44c6adb7a45ce093b5fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:39.641538 27216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39970ef3d54e44c6adb7a45ce093b5fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:39.643543 27427 tablet_bootstrap.cc:492] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34: Bootstrap starting.
I20260926 14:59:39.644124 27427 tablet_bootstrap.cc:654] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:39.644613 27427 log.cc:824] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:39.644764 27428 tablet_bootstrap.cc:492] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0: Bootstrap starting.
I20260926 14:59:39.645255 27427 tablet_bootstrap.cc:492] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34: No bootstrap required, opened a new log
I20260926 14:59:39.645289 27428 tablet_bootstrap.cc:654] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:39.645310 27427 ts_tablet_manager.cc:1397] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:59:39.645526 27428 log.cc:824] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:39.645773 27429 tablet_bootstrap.cc:492] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb: Bootstrap starting.
I20260926 14:59:39.646209 27428 tablet_bootstrap.cc:492] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0: No bootstrap required, opened a new log
I20260926 14:59:39.646219 27429 tablet_bootstrap.cc:654] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:39.646307 27428 ts_tablet_manager.cc:1397] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 14:59:39.646445 27429 log.cc:824] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:39.646996 27429 tablet_bootstrap.cc:492] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb: No bootstrap required, opened a new log
I20260926 14:59:39.647044 27429 ts_tablet_manager.cc:1397] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 14:59:39.647616 27427 raft_consensus.cc:348] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7211846b090478cadeefd1a491e57b0" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42401 } } peers { permanent_uuid: "d9032abd3e5b42fb8ccad918ab3abfdb" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35515 } } peers { permanent_uuid: "90046ca8651f4494967f7ad3d985ac34" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45193 } }
I20260926 14:59:39.647794 27427 raft_consensus.cc:374] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:39.647841 27427 raft_consensus.cc:729] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90046ca8651f4494967f7ad3d985ac34, State: Initialized, Role: FOLLOWER
I20260926 14:59:39.647949 27427 consensus_queue.cc:260] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7211846b090478cadeefd1a491e57b0" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42401 } } peers { permanent_uuid: "d9032abd3e5b42fb8ccad918ab3abfdb" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35515 } } peers { permanent_uuid: "90046ca8651f4494967f7ad3d985ac34" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45193 } }
I20260926 14:59:39.648204 27427 ts_tablet_manager.cc:1428] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 14:59:39.648340 27151 heartbeater.cc:503] Master 127.7.37.254:36523 was elected leader, sending a full tablet report...
I20260926 14:59:39.648922 27428 raft_consensus.cc:348] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7211846b090478cadeefd1a491e57b0" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42401 } } peers { permanent_uuid: "d9032abd3e5b42fb8ccad918ab3abfdb" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35515 } } peers { permanent_uuid: "90046ca8651f4494967f7ad3d985ac34" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45193 } }
I20260926 14:59:39.649066 27428 raft_consensus.cc:374] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:39.649091 27428 raft_consensus.cc:729] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7211846b090478cadeefd1a491e57b0, State: Initialized, Role: FOLLOWER
I20260926 14:59:39.649169 27428 consensus_queue.cc:260] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7211846b090478cadeefd1a491e57b0" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42401 } } peers { permanent_uuid: "d9032abd3e5b42fb8ccad918ab3abfdb" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35515 } } peers { permanent_uuid: "90046ca8651f4494967f7ad3d985ac34" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45193 } }
I20260926 14:59:39.649111 27429 raft_consensus.cc:348] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7211846b090478cadeefd1a491e57b0" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42401 } } peers { permanent_uuid: "d9032abd3e5b42fb8ccad918ab3abfdb" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35515 } } peers { permanent_uuid: "90046ca8651f4494967f7ad3d985ac34" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45193 } }
I20260926 14:59:39.649257 27429 raft_consensus.cc:374] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:39.649296 27429 raft_consensus.cc:729] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9032abd3e5b42fb8ccad918ab3abfdb, State: Initialized, Role: FOLLOWER
I20260926 14:59:39.649380 27428 ts_tablet_manager.cc:1428] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 14:59:39.649401 27429 consensus_queue.cc:260] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7211846b090478cadeefd1a491e57b0" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42401 } } peers { permanent_uuid: "d9032abd3e5b42fb8ccad918ab3abfdb" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35515 } } peers { permanent_uuid: "90046ca8651f4494967f7ad3d985ac34" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45193 } }
I20260926 14:59:39.649434 27282 heartbeater.cc:503] Master 127.7.37.254:36523 was elected leader, sending a full tablet report...
I20260926 14:59:39.649626 27429 ts_tablet_manager.cc:1428] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 14:59:39.649734 27413 heartbeater.cc:503] Master 127.7.37.254:36523 was elected leader, sending a full tablet report...
I20260926 14:59:39.670517 27433 raft_consensus.cc:482] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 14:59:39.670581 27433 raft_consensus.cc:504] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7211846b090478cadeefd1a491e57b0" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42401 } } peers { permanent_uuid: "d9032abd3e5b42fb8ccad918ab3abfdb" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35515 } } peers { permanent_uuid: "90046ca8651f4494967f7ad3d985ac34" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45193 } }
I20260926 14:59:39.670842 27433 leader_election.cc:288] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7211846b090478cadeefd1a491e57b0 (127.7.37.194:42401), d9032abd3e5b42fb8ccad918ab3abfdb (127.7.37.195:35515)
I20260926 14:59:39.673370 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.670991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:33845 (local address 127.7.37.194:42401)
0926 14:59:39.671031 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:39.671033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:39.671137 (+ 104us) server_negotiation.cc:407] Connection header received
0926 14:59:39.671163 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:39.671164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:39.671172 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:39.671223 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:39.671356 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.671906 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.672754 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.673098 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.673154 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.673161 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:39.673285 (+ 124us) server_negotiation.cc:299] Negotiation successful
0926 14:59:39.673314 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 14:59:39.673373 27153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.670952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.194:42401 (local address 127.7.37.193:33845)
0926 14:59:39.671020 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:39.671031 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:39.671077 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:39.671245 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:39.671246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:39.671248 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:39.671300 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.671302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.671973 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.671974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.672683 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.672685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.673176 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.673177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.673231 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.673233 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 14:59:39.673275 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 14:59:39.673312 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260926 14:59:39.673470 27426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.671095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.37.193:58817 (local address 127.7.37.195:35515)
0926 14:59:39.671127 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:39.671128 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:39.671241 (+ 113us) server_negotiation.cc:407] Connection header received
0926 14:59:39.671284 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:39.671284 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:39.671292 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:39.671335 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 14:59:39.671476 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.671972 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.672852 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.673187 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.673242 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.673248 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:39.673388 (+ 140us) server_negotiation.cc:299] Negotiation successful
0926 14:59:39.673418 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 14:59:39.673480 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.671041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.195:35515 (local address 127.7.37.193:58817)
0926 14:59:39.671164 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:39.671172 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 14:59:39.671233 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 14:59:39.671367 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 14:59:39.671368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 14:59:39.671369 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 14:59:39.671422 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.671424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.672037 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.672038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.672774 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 14:59:39.672776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.673257 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.673257 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 14:59:39.673330 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.673333 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 14:59:39.673376 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 14:59:39.673408 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
I20260926 14:59:39.673638 27236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39970ef3d54e44c6adb7a45ce093b5fd" candidate_uuid: "90046ca8651f4494967f7ad3d985ac34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7211846b090478cadeefd1a491e57b0" is_pre_election: true
I20260926 14:59:39.673745 27236 raft_consensus.cc:2454] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90046ca8651f4494967f7ad3d985ac34 in term 0.
I20260926 14:59:39.673816 27367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39970ef3d54e44c6adb7a45ce093b5fd" candidate_uuid: "90046ca8651f4494967f7ad3d985ac34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9032abd3e5b42fb8ccad918ab3abfdb" is_pre_election: true
I20260926 14:59:39.673923 27367 raft_consensus.cc:2454] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90046ca8651f4494967f7ad3d985ac34 in term 0.
I20260926 14:59:39.673925 27040 leader_election.cc:302] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90046ca8651f4494967f7ad3d985ac34, f7211846b090478cadeefd1a491e57b0; no voters:
I20260926 14:59:39.674021 27433 raft_consensus.cc:2781] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 14:59:39.674070 27433 raft_consensus.cc:482] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 14:59:39.674085 27433 raft_consensus.cc:3037] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:39.674628 27433 raft_consensus.cc:504] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7211846b090478cadeefd1a491e57b0" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42401 } } peers { permanent_uuid: "d9032abd3e5b42fb8ccad918ab3abfdb" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35515 } } peers { permanent_uuid: "90046ca8651f4494967f7ad3d985ac34" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45193 } }
I20260926 14:59:39.674718 27433 leader_election.cc:288] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [CANDIDATE]: Term 1 election: Requested vote from peers f7211846b090478cadeefd1a491e57b0 (127.7.37.194:42401), d9032abd3e5b42fb8ccad918ab3abfdb (127.7.37.195:35515)
I20260926 14:59:39.674878 27236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39970ef3d54e44c6adb7a45ce093b5fd" candidate_uuid: "90046ca8651f4494967f7ad3d985ac34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7211846b090478cadeefd1a491e57b0"
I20260926 14:59:39.674906 27367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39970ef3d54e44c6adb7a45ce093b5fd" candidate_uuid: "90046ca8651f4494967f7ad3d985ac34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9032abd3e5b42fb8ccad918ab3abfdb"
I20260926 14:59:39.674944 27236 raft_consensus.cc:3037] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:39.674961 27367 raft_consensus.cc:3037] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:39.675508 27367 raft_consensus.cc:2454] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90046ca8651f4494967f7ad3d985ac34 in term 1.
I20260926 14:59:39.675503 27236 raft_consensus.cc:2454] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90046ca8651f4494967f7ad3d985ac34 in term 1.
I20260926 14:59:39.675644 27040 leader_election.cc:302] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90046ca8651f4494967f7ad3d985ac34, d9032abd3e5b42fb8ccad918ab3abfdb; no voters:
I20260926 14:59:39.675720 27433 raft_consensus.cc:2781] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:39.675879 27433 raft_consensus.cc:686] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [term 1 LEADER]: Becoming Leader. State: Replica: 90046ca8651f4494967f7ad3d985ac34, State: Running, Role: LEADER
I20260926 14:59:39.675951 27433 consensus_queue.cc:237] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7211846b090478cadeefd1a491e57b0" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42401 } } peers { permanent_uuid: "d9032abd3e5b42fb8ccad918ab3abfdb" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35515 } } peers { permanent_uuid: "90046ca8651f4494967f7ad3d985ac34" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45193 } }
I20260926 14:59:39.676573 26962 catalog_manager.cc:5495] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90046ca8651f4494967f7ad3d985ac34 (127.7.37.193). New cstate: current_term: 1 leader_uuid: "90046ca8651f4494967f7ad3d985ac34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7211846b090478cadeefd1a491e57b0" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9032abd3e5b42fb8ccad918ab3abfdb" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90046ca8651f4494967f7ad3d985ac34" member_type: VOTER last_known_addr { host: "127.7.37.193" port: 45193 } health_report { overall_health: HEALTHY } } }
I20260926 14:59:39.684476 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.682331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49280 (local address 127.7.37.193:45193)
0926 14:59:39.682371 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:39.682373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:39.682382 (+ 9us) server_negotiation.cc:407] Connection header received
0926 14:59:39.682452 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:39.682453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:39.682470 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:39.682534 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 14:59:39.682647 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.683080 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.683788 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:39.683981 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:39.684033 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:39.684187 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 14:59:39.684266 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 14:59:39.684332 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:39.684380 (+ 48us) server_negotiation.cc:299] Negotiation successful
0926 14:59:39.684397 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 14:59:39.685756 27367 raft_consensus.cc:1261] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb [term 1 FOLLOWER]: Refusing update from remote peer 90046ca8651f4494967f7ad3d985ac34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:59:39.685761 27236 raft_consensus.cc:1261] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0 [term 1 FOLLOWER]: Refusing update from remote peer 90046ca8651f4494967f7ad3d985ac34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 14:59:39.686057 27437 consensus_queue.cc:1035] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7211846b090478cadeefd1a491e57b0" member_type: VOTER last_known_addr { host: "127.7.37.194" port: 42401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:59:39.686174 27433 consensus_queue.cc:1035] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9032abd3e5b42fb8ccad918ab3abfdb" member_type: VOTER last_known_addr { host: "127.7.37.195" port: 35515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 14:59:39.688467 27442 mvcc.cc:205] Tried to move back new op lower bound from 7333620857591054336 to 7333620857553223680. Current Snapshot: MvccSnapshot[applied={T|T < 7333620857591054336}]
I20260926 14:59:39.688688 27441 mvcc.cc:205] Tried to move back new op lower bound from 7333620857591054336 to 7333620857553223680. Current Snapshot: MvccSnapshot[applied={T|T < 7333620857591054336}]
I20260926 14:59:39.690061 26962 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:59:39.690136 26962 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 14:59:39.690703 26962 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f85049d6f889448e84ba8b941a893d73: Sending DeleteTablet for 3 replicas of tablet 39970ef3d54e44c6adb7a45ce093b5fd
I20260926 14:59:39.691004 27085 tablet_service.cc:1514] Processing DeleteTablet for tablet 39970ef3d54e44c6adb7a45ce093b5fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:59:39 UTC) from {username='slave'} at 127.0.0.1:49268
I20260926 14:59:39.691090 27216 tablet_service.cc:1514] Processing DeleteTablet for tablet 39970ef3d54e44c6adb7a45ce093b5fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:59:39 UTC) from {username='slave'} at 127.0.0.1:44962
I20260926 14:59:39.691232 27450 tablet_replica.cc:330] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34: stopping tablet replica
I20260926 14:59:39.691212 27347 tablet_service.cc:1514] Processing DeleteTablet for tablet 39970ef3d54e44c6adb7a45ce093b5fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 14:59:39 UTC) from {username='slave'} at 127.0.0.1:50098
I20260926 14:59:39.691314 27450 raft_consensus.cc:2229] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [term 1 LEADER]: Raft consensus shutting down.
I20260926 14:59:39.691399 27450 raft_consensus.cc:2258] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:59:39.691464 27452 tablet_replica.cc:330] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb: stopping tablet replica
I20260926 14:59:39.691583 27452 raft_consensus.cc:2229] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:59:39.691669 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 27023
I20260926 14:59:39.691772 27452 raft_consensus.cc:2258] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:59:39.691857 27450 ts_tablet_manager.cc:1905] T 39970ef3d54e44c6adb7a45ce093b5fd P 90046ca8651f4494967f7ad3d985ac34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:59:39.693418 27452 ts_tablet_manager.cc:1905] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:59:39.694123 27451 tablet_replica.cc:330] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0: stopping tablet replica
I20260926 14:59:39.694211 27451 raft_consensus.cc:2229] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 14:59:39.694300 27451 raft_consensus.cc:2258] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 14:59:39.694700 27451 ts_tablet_manager.cc:1905] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 14:59:39.694975 27452 ts_tablet_manager.cc:1918] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:59:39.695042 27452 log.cc:1196] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-2/wal/wals/39970ef3d54e44c6adb7a45ce093b5fd
I20260926 14:59:39.695279 27452 ts_tablet_manager.cc:1939] T 39970ef3d54e44c6adb7a45ce093b5fd P d9032abd3e5b42fb8ccad918ab3abfdb: Deleting consensus metadata
I20260926 14:59:39.695634 26949 catalog_manager.cc:4841] TS d9032abd3e5b42fb8ccad918ab3abfdb (127.7.37.195:35515): tablet 39970ef3d54e44c6adb7a45ce093b5fd (table test-table [id=059fecb1190f4066af3b51bf6a07a6eb]) successfully deleted
I20260926 14:59:39.696146 27451 ts_tablet_manager.cc:1918] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 14:59:39.696220 27451 log.cc:1196] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0: Deleting WAL directory at /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434555014799-7319-0/minicluster-data/ts-1/wal/wals/39970ef3d54e44c6adb7a45ce093b5fd
I20260926 14:59:39.696480 27451 ts_tablet_manager.cc:1939] T 39970ef3d54e44c6adb7a45ce093b5fd P f7211846b090478cadeefd1a491e57b0: Deleting consensus metadata
I20260926 14:59:39.696841 26949 catalog_manager.cc:4841] TS f7211846b090478cadeefd1a491e57b0 (127.7.37.194:42401): tablet 39970ef3d54e44c6adb7a45ce093b5fd (table test-table [id=059fecb1190f4066af3b51bf6a07a6eb]) successfully deleted
W20260926 14:59:39.697556 26948 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.37.193:45193 (error 108)
I20260926 14:59:39.697790 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 27154
I20260926 14:59:39.697948 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:39.697845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.193:45193 (local address 127.0.0.1:49296)
0926 14:59:39.697889 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 14:59:39.697921 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.193:45193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 14:59:39.698390 26948 catalog_manager.cc:4584] TS 90046ca8651f4494967f7ad3d985ac34 (127.7.37.193:45193): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 39970ef3d54e44c6adb7a45ce093b5fd: Network error: Client connection negotiation failed: client connection to 127.7.37.193:45193: connect: Connection refused (error 111)
I20260926 14:59:39.701900 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 27285
I20260926 14:59:39.705797 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 26932
2026-09-26T14:59:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (630 ms)
[----------] 6 tests from EncryptionPolicyTest (7410 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:39 dist-test-slave-f9xb krb5kdc[27456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:39 dist-test-slave-f9xb krb5kdc[27456](info): set up 2 sockets
Sep 26 14:59:39 dist-test-slave-f9xb krb5kdc[27456](info): commencing operation
krb5kdc: starting...
W20260926 14:59:40.651357 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.928s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:40 dist-test-slave-f9xb krb5kdc[27456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 14:59:40.754347 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46717
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:40.830116 27469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:40.830293 27469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:40.830322 27469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:40.831408 27469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:40.831446 27469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:40.831463 27469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:40.831480 27469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:40.831496 27469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:40.832917 27469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46717
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:40.833127 27469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:40.833307 27469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 14:59:40.837568 27469 server_base.cc:1034] running on GCE node
W20260926 14:59:40.837544 27474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:40.837612 27475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:40.837545 27477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:40.837910 27469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:40.838145 27469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:40.839268 27469 hybrid_clock.cc:648] HybridClock initialized: now 1790434780839256 us; error 26 us; skew 500 ppm
Sep 26 14:59:40 dist-test-slave-f9xb krb5kdc[27456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:40.841445 27469 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 14:59:40.841830 27469 webserver.cc:460] Webserver started at http://127.0.0.1:35393/ using document root <none> and password file <none>
I20260926 14:59:40.842017 27469 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:40.842061 27469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:40.842175 27469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:40.843066 27469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "63cc98ac9b744084b113221f7d77db56"
format_stamp: "Formatted at 2026-09-26 14:59:40 on dist-test-slave-f9xb"
server_key: "34af6b6947e764b28817fa5bd6e9139b"
server_key_iv: "494feb300627d6fb6c4e4db12daa9dec"
server_key_version: "encryptionkey@0"
I20260926 14:59:40.843357 27469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "63cc98ac9b744084b113221f7d77db56"
format_stamp: "Formatted at 2026-09-26 14:59:40 on dist-test-slave-f9xb"
server_key: "34af6b6947e764b28817fa5bd6e9139b"
server_key_iv: "494feb300627d6fb6c4e4db12daa9dec"
server_key_version: "encryptionkey@0"
I20260926 14:59:40.844503 27469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 14:59:40.845100 27484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:40.845238 27469 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:59:40.845304 27469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "63cc98ac9b744084b113221f7d77db56"
format_stamp: "Formatted at 2026-09-26 14:59:40 on dist-test-slave-f9xb"
server_key: "34af6b6947e764b28817fa5bd6e9139b"
server_key_iv: "494feb300627d6fb6c4e4db12daa9dec"
server_key_version: "encryptionkey@0"
I20260926 14:59:40.845359 27469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:40.860203 27469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:40.862015 27469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:40.862131 27469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:40.865520 27469 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46717
I20260926 14:59:40.865554 27536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46717 every 8 connection(s)
I20260926 14:59:40.865828 27469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:40.866618 27537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:40.867652 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 27469
I20260926 14:59:40.867787 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:40.867982 7319 external_mini_cluster.cc:1428] Setting key 1e8541436dcd4e98a23dd071fcc339b1
I20260926 14:59:40.868877 27537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56: Bootstrap starting.
I20260926 14:59:40.869467 27537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:40.869729 27537 log.cc:824] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:40.870363 27537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56: No bootstrap required, opened a new log
I20260926 14:59:40.872382 27537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cc98ac9b744084b113221f7d77db56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46717 } }
I20260926 14:59:40.872535 27537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:40.872567 27537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63cc98ac9b744084b113221f7d77db56, State: Initialized, Role: FOLLOWER
I20260926 14:59:40.872712 27537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cc98ac9b744084b113221f7d77db56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46717 } }
I20260926 14:59:40.872799 27537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:40.872833 27537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:40.872875 27537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:40.873448 27537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cc98ac9b744084b113221f7d77db56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46717 } }
I20260926 14:59:40.873539 27537 leader_election.cc:302] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63cc98ac9b744084b113221f7d77db56; no voters:
I20260926 14:59:40.873684 27537 leader_election.cc:288] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:40.873732 27542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:40.873893 27537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:40.873919 27542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56 [term 1 LEADER]: Becoming Leader. State: Replica: 63cc98ac9b744084b113221f7d77db56, State: Running, Role: LEADER
I20260926 14:59:40.874090 27542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cc98ac9b744084b113221f7d77db56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46717 } }
I20260926 14:59:40.874445 27544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63cc98ac9b744084b113221f7d77db56. Latest consensus state: current_term: 1 leader_uuid: "63cc98ac9b744084b113221f7d77db56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cc98ac9b744084b113221f7d77db56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46717 } } }
I20260926 14:59:40.874470 27543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63cc98ac9b744084b113221f7d77db56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cc98ac9b744084b113221f7d77db56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46717 } } }
I20260926 14:59:40.874536 27543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56 [sys.catalog]: This master's current role is: LEADER
Sep 26 14:59:40 dist-test-slave-f9xb krb5kdc[27456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 14:59:40.874523 27544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:40.874790 27551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:40.875268 27551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:40.877147 27551 catalog_manager.cc:1347] Generated new cluster ID: 11f4fbc6f98e417ca7f7e5a0c2f8c9cf
I20260926 14:59:40.877195 27551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:59:40.878690 27540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:40.868477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44435 (local address 127.0.0.1:46717)
0926 14:59:40.868635 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:40.868643 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:40.868672 (+ 29us) server_negotiation.cc:407] Connection header received
0926 14:59:40.869128 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:40.869137 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:40.869209 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:40.869351 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:40.869913 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:40.871185 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:40.872003 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:40.872608 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:40.872710 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:40.875129 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:40.875133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:40.875133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:40.875144 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:40.877098 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:40.877513 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:40.877515 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:40.877516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:40.877613 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:40.877812 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:40.877812 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:40.877813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:40.878053 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:40.878166 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:40.878400 (+ 234us) server_negotiation.cc:299] Negotiation successful
0926 14:59:40.878488 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
I20260926 14:59:40.879801 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:59:40.887544 27540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:40.881064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48771 (local address 127.0.0.1:46717)
0926 14:59:40.881097 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:40.881099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:40.881187 (+ 88us) server_negotiation.cc:407] Connection header received
0926 14:59:40.881251 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:40.881251 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:40.881259 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:40.881303 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:40.881885 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:40.882554 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:40.883507 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:40.883830 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:40.883883 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:40.885095 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:40.885096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:40.885097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:40.885103 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:40.886492 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:40.886869 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:40.886869 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:40.886870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:40.886915 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:40.887101 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:40.887101 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:40.887102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:40.887203 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:40.887291 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:40.887414 (+ 123us) server_negotiation.cc:299] Negotiation successful
0926 14:59:40.887468 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 14:59:40.888379 27551 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:40.888824 27551 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:40.893306 27551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63cc98ac9b744084b113221f7d77db56: Generated new TSK 0
I20260926 14:59:40.893453 27551 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:40.896641 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 27469
W20260926 14:59:40.900069 27454 connection.cc:535] client connection to 127.0.0.1:46717 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46717): Connection reset by peer (error 104)
2026-09-26T14:59:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1189 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:40 dist-test-slave-f9xb krb5kdc[27568](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:40 dist-test-slave-f9xb krb5kdc[27568](info): set up 2 sockets
Sep 26 14:59:40 dist-test-slave-f9xb krb5kdc[27568](info): commencing operation
krb5kdc: starting...
W20260926 14:59:41.831420 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.918s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:41 dist-test-slave-f9xb krb5kdc[27568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 14:59:41.944726 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36291
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:42.021472 27581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:42.021677 27581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:42.021708 27581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:42.022832 27581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:42.022871 27581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:42.022891 27581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:42.022905 27581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:42.022921 27581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:42.024402 27581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36291
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:42.024631 27581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:42.024816 27581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:42.029184 27586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:42.029198 27589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:42.029250 27587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:42.029281 27581 server_base.cc:1034] running on GCE node
I20260926 14:59:42.029594 27581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:42.029872 27581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:42.031005 27581 hybrid_clock.cc:648] HybridClock initialized: now 1790434782030990 us; error 32 us; skew 500 ppm
Sep 26 14:59:42 dist-test-slave-f9xb krb5kdc[27568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434782, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:42.033277 27581 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 14:59:42.033664 27581 webserver.cc:460] Webserver started at http://127.0.0.1:34405/ using document root <none> and password file <none>
I20260926 14:59:42.033846 27581 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:42.033891 27581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:42.034010 27581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:42.034962 27581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "e7d3c3a55d484007bbe9019e99d7112e"
format_stamp: "Formatted at 2026-09-26 14:59:42 on dist-test-slave-f9xb"
server_key: "ad19db1880a1da8d880c7e94e999b45b"
server_key_iv: "d2af4fc742fbc4d25250219d2f400b0d"
server_key_version: "encryptionkey@0"
I20260926 14:59:42.035269 27581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "e7d3c3a55d484007bbe9019e99d7112e"
format_stamp: "Formatted at 2026-09-26 14:59:42 on dist-test-slave-f9xb"
server_key: "ad19db1880a1da8d880c7e94e999b45b"
server_key_iv: "d2af4fc742fbc4d25250219d2f400b0d"
server_key_version: "encryptionkey@0"
I20260926 14:59:42.036518 27581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 14:59:42.037231 27596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:42.037388 27581 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:42.037467 27581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "e7d3c3a55d484007bbe9019e99d7112e"
format_stamp: "Formatted at 2026-09-26 14:59:42 on dist-test-slave-f9xb"
server_key: "ad19db1880a1da8d880c7e94e999b45b"
server_key_iv: "d2af4fc742fbc4d25250219d2f400b0d"
server_key_version: "encryptionkey@0"
I20260926 14:59:42.037521 27581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:42.056653 27581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:42.058630 27581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:42.058758 27581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:42.062290 27581 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36291
I20260926 14:59:42.062323 27648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36291 every 8 connection(s)
I20260926 14:59:42.062625 27581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:42.063512 27649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:42.065623 27649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e: Bootstrap starting.
I20260926 14:59:42.066134 27649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:42.066370 27649 log.cc:824] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:42.066958 27649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e: No bootstrap required, opened a new log
I20260926 14:59:42.068562 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 27581
I20260926 14:59:42.068557 27649 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d3c3a55d484007bbe9019e99d7112e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36291 } }
I20260926 14:59:42.068691 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:42.068722 27649 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:42.068760 27649 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7d3c3a55d484007bbe9019e99d7112e, State: Initialized, Role: FOLLOWER
I20260926 14:59:42.068890 7319 external_mini_cluster.cc:1428] Setting key 8733f132aa8bf0a7a22654bec3b39e71
I20260926 14:59:42.068899 27649 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d3c3a55d484007bbe9019e99d7112e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36291 } }
I20260926 14:59:42.068972 27649 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:42.069006 27649 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:42.069053 27649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:42.069656 27649 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d3c3a55d484007bbe9019e99d7112e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36291 } }
I20260926 14:59:42.069746 27649 leader_election.cc:302] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7d3c3a55d484007bbe9019e99d7112e; no voters:
I20260926 14:59:42.069901 27649 leader_election.cc:288] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:42.069936 27654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:42.070061 27654 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e [term 1 LEADER]: Becoming Leader. State: Replica: e7d3c3a55d484007bbe9019e99d7112e, State: Running, Role: LEADER
I20260926 14:59:42.070137 27649 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:42.070183 27654 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d3c3a55d484007bbe9019e99d7112e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36291 } }
I20260926 14:59:42.070540 27656 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7d3c3a55d484007bbe9019e99d7112e. Latest consensus state: current_term: 1 leader_uuid: "e7d3c3a55d484007bbe9019e99d7112e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d3c3a55d484007bbe9019e99d7112e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36291 } } }
I20260926 14:59:42.070536 27655 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7d3c3a55d484007bbe9019e99d7112e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d3c3a55d484007bbe9019e99d7112e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36291 } } }
I20260926 14:59:42.070638 27656 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:42.070641 27655 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:42.070911 27662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:42.071571 27662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:42.073026 27662 catalog_manager.cc:1347] Generated new cluster ID: b3a685d477964e6287c85b48808b3333
I20260926 14:59:42.073074 27662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:59:42.074646 27653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:42.069355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52171 (local address 127.0.0.1:36291)
0926 14:59:42.069795 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:42.069803 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:42.069847 (+ 44us) server_negotiation.cc:407] Connection header received
0926 14:59:42.069957 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:42.069963 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:42.070019 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:42.070381 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:42.070467 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:42.071853 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:42.072887 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:42.073351 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:42.073445 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:42.074052 (+ 607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:42.074060 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:42.074060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:42.074221 (+ 161us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:42.074249 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:42.074260 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:42.074326 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:42.074416 (+ 90us) server_negotiation.cc:299] Negotiation successful
0926 14:59:42.074446 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":341,"threads_started":1}
I20260926 14:59:42.075435 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:59:42.079139 27653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:42.076659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56529 (local address 127.0.0.1:36291)
0926 14:59:42.076729 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:42.076731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:42.076771 (+ 40us) server_negotiation.cc:407] Connection header received
0926 14:59:42.076823 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:42.076823 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:42.076844 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:42.076893 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:42.077022 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:42.077469 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:42.078122 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:42.078346 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:42.078396 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:42.078932 (+ 536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:42.078933 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:42.078933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:42.078937 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:42.078950 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:42.078954 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:42.079000 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:42.079061 (+ 61us) server_negotiation.cc:299] Negotiation successful
0926 14:59:42.079072 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 14:59:42.093607 27662 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:42.094087 27662 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:42.099004 27662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7d3c3a55d484007bbe9019e99d7112e: Generated new TSK 0
I20260926 14:59:42.099155 27662 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:42.100767 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 27581
W20260926 14:59:42.104387 27566 connection.cc:535] client connection to 127.0.0.1:36291 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36291): Connection reset by peer (error 104)
2026-09-26T14:59:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1204 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:42 dist-test-slave-f9xb krb5kdc[27680](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:42 dist-test-slave-f9xb krb5kdc[27680](info): set up 2 sockets
Sep 26 14:59:42 dist-test-slave-f9xb krb5kdc[27680](info): commencing operation
krb5kdc: starting...
W20260926 14:59:43.052099 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:43 dist-test-slave-f9xb krb5kdc[27680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 14:59:43.155884 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41589
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:43.230988 27693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:43.231160 27693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:43.231189 27693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:43.232311 27693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:43.232350 27693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:43.232367 27693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:43.232383 27693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:43.232398 27693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:43.233772 27693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41589
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:43.233981 27693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:43.234164 27693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:43.238487 27701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:43.238493 27699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:43.238494 27698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:43.238523 27693 server_base.cc:1034] running on GCE node
I20260926 14:59:43.238883 27693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:43.239122 27693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:43.240268 27693 hybrid_clock.cc:648] HybridClock initialized: now 1790434783240251 us; error 34 us; skew 500 ppm
Sep 26 14:59:43 dist-test-slave-f9xb krb5kdc[27680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434783, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:43.246428 27693 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 14:59:43.246824 27693 webserver.cc:460] Webserver started at http://127.0.0.1:45223/ using document root <none> and password file <none>
I20260926 14:59:43.247014 27693 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:43.247059 27693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:43.247179 27693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:43.248167 27693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "ee0b890b1e374a05a8e4cf32a76bfde5"
format_stamp: "Formatted at 2026-09-26 14:59:43 on dist-test-slave-f9xb"
server_key: "a018643e7678ed908709e8cf44b43131"
server_key_iv: "9813a239d06ae8dc6919d689a547076d"
server_key_version: "encryptionkey@0"
I20260926 14:59:43.248461 27693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "ee0b890b1e374a05a8e4cf32a76bfde5"
format_stamp: "Formatted at 2026-09-26 14:59:43 on dist-test-slave-f9xb"
server_key: "a018643e7678ed908709e8cf44b43131"
server_key_iv: "9813a239d06ae8dc6919d689a547076d"
server_key_version: "encryptionkey@0"
I20260926 14:59:43.249620 27693 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:43.250242 27708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:43.250396 27693 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:43.250473 27693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "ee0b890b1e374a05a8e4cf32a76bfde5"
format_stamp: "Formatted at 2026-09-26 14:59:43 on dist-test-slave-f9xb"
server_key: "a018643e7678ed908709e8cf44b43131"
server_key_iv: "9813a239d06ae8dc6919d689a547076d"
server_key_version: "encryptionkey@0"
I20260926 14:59:43.250526 27693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:43.253564 27693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:43.255486 27693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:43.255595 27693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:43.259099 27693 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41589
I20260926 14:59:43.259127 27760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41589 every 8 connection(s)
I20260926 14:59:43.259389 27693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:43.260268 27761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:43.262228 27761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5: Bootstrap starting.
I20260926 14:59:43.262728 27761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:43.262969 27761 log.cc:824] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:43.263571 27761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5: No bootstrap required, opened a new log
I20260926 14:59:43.265089 27761 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee0b890b1e374a05a8e4cf32a76bfde5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41589 } }
I20260926 14:59:43.265218 27761 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:43.265250 27761 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee0b890b1e374a05a8e4cf32a76bfde5, State: Initialized, Role: FOLLOWER
I20260926 14:59:43.265379 27761 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee0b890b1e374a05a8e4cf32a76bfde5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41589 } }
I20260926 14:59:43.265450 27761 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:43.265487 27761 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:43.265530 27761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:43.265998 27761 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee0b890b1e374a05a8e4cf32a76bfde5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41589 } }
I20260926 14:59:43.266083 27761 leader_election.cc:302] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee0b890b1e374a05a8e4cf32a76bfde5; no voters:
I20260926 14:59:43.266224 27761 leader_election.cc:288] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:43.266258 27764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:43.266439 27761 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:43.266541 27764 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5 [term 1 LEADER]: Becoming Leader. State: Replica: ee0b890b1e374a05a8e4cf32a76bfde5, State: Running, Role: LEADER
I20260926 14:59:43.266625 27764 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee0b890b1e374a05a8e4cf32a76bfde5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41589 } }
I20260926 14:59:43.266943 27765 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee0b890b1e374a05a8e4cf32a76bfde5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee0b890b1e374a05a8e4cf32a76bfde5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41589 } } }
I20260926 14:59:43.267036 27765 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:43.266952 27766 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee0b890b1e374a05a8e4cf32a76bfde5. Latest consensus state: current_term: 1 leader_uuid: "ee0b890b1e374a05a8e4cf32a76bfde5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee0b890b1e374a05a8e4cf32a76bfde5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41589 } } }
I20260926 14:59:43.267230 27766 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:43.267211 27773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:43.267822 27773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:43.269134 27773 catalog_manager.cc:1347] Generated new cluster ID: f0319e78979c427b9f090c862c7caf11
I20260926 14:59:43.269207 27773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:59:43.269234 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 27693
I20260926 14:59:43.269353 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:43.269533 7319 external_mini_cluster.cc:1428] Setting key 8a324e145c52c7baad23c2e56e9e1b1b
I20260926 14:59:43.277351 27784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:43.269929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48291 (local address 127.0.0.1:41589)
0926 14:59:43.276070 (+ 6141us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:43.276078 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:43.276108 (+ 30us) server_negotiation.cc:407] Connection header received
0926 14:59:43.276213 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:43.276219 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:43.276247 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:43.276366 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:43.276924 (+ 558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:43.276928 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:43.276928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:43.276931 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:43.276966 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:43.276979 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:43.277037 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:43.277105 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 14:59:43.277144 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6088,"thread_start_us":69,"threads_started":1}
I20260926 14:59:43.278160 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:59:43.279886 27773 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:43.280015 27773 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:43.283910 27773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee0b890b1e374a05a8e4cf32a76bfde5: Generated new TSK 0
I20260926 14:59:43.284101 27773 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:43.284787 27784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:43.280032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35629 (local address 127.0.0.1:41589)
0926 14:59:43.280060 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:43.280061 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:43.283895 (+ 3834us) server_negotiation.cc:407] Connection header received
0926 14:59:43.283915 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:43.283916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:43.283920 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:43.283960 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:43.284600 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:43.284601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:43.284601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:43.284605 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:43.284619 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:43.284623 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:43.284689 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:43.284722 (+ 33us) server_negotiation.cc:299] Negotiation successful
0926 14:59:43.284733 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260926 14:59:43.285913 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 27693
2026-09-26T14:59:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1185 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:43 dist-test-slave-f9xb krb5kdc[27792](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:43 dist-test-slave-f9xb krb5kdc[27792](info): set up 2 sockets
Sep 26 14:59:43 dist-test-slave-f9xb krb5kdc[27792](info): commencing operation
krb5kdc: starting...
W20260926 14:59:44.239395 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:44 dist-test-slave-f9xb krb5kdc[27792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 14:59:44.341694 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43431
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:44.416193 27805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:44.416363 27805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:44.416383 27805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:44.417483 27805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:44.417519 27805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:44.417532 27805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:44.417544 27805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:44.417567 27805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:44.418938 27805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:44.419135 27805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:44.419318 27805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:44.423513 27811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:44.423508 27813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:44.423595 27810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:44.423602 27805 server_base.cc:1034] running on GCE node
I20260926 14:59:44.423941 27805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:44.424196 27805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:44.425350 27805 hybrid_clock.cc:648] HybridClock initialized: now 1790434784425335 us; error 24 us; skew 500 ppm
Sep 26 14:59:44 dist-test-slave-f9xb krb5kdc[27792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434784, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:44.433092 27805 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 14:59:44.433506 27805 webserver.cc:460] Webserver started at http://127.0.0.1:37629/ using document root <none> and password file <none>
I20260926 14:59:44.433686 27805 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:44.433730 27805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:44.433842 27805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:44.434779 27805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "64ef0a66ffec41ea9fe22c7a5d9b3532"
format_stamp: "Formatted at 2026-09-26 14:59:44 on dist-test-slave-f9xb"
server_key: "dea2abf9e9ce00f0ed9d0b4b5e247a4e"
server_key_iv: "a56dd301b5df7f86874f0d57b419b56e"
server_key_version: "encryptionkey@0"
I20260926 14:59:44.435071 27805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "64ef0a66ffec41ea9fe22c7a5d9b3532"
format_stamp: "Formatted at 2026-09-26 14:59:44 on dist-test-slave-f9xb"
server_key: "dea2abf9e9ce00f0ed9d0b4b5e247a4e"
server_key_iv: "a56dd301b5df7f86874f0d57b419b56e"
server_key_version: "encryptionkey@0"
I20260926 14:59:44.436265 27805 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 14:59:44.436849 27820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:44.436990 27805 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:44.437062 27805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "64ef0a66ffec41ea9fe22c7a5d9b3532"
format_stamp: "Formatted at 2026-09-26 14:59:44 on dist-test-slave-f9xb"
server_key: "dea2abf9e9ce00f0ed9d0b4b5e247a4e"
server_key_iv: "a56dd301b5df7f86874f0d57b419b56e"
server_key_version: "encryptionkey@0"
I20260926 14:59:44.437117 27805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:44.450006 27805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:44.451445 27805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:44.451556 27805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:44.455130 27805 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43431
I20260926 14:59:44.455159 27872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43431 every 8 connection(s)
I20260926 14:59:44.455426 27805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:44.456285 27873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:44.458199 27873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532: Bootstrap starting.
I20260926 14:59:44.458655 27873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:44.458863 27873 log.cc:824] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:44.459421 27873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532: No bootstrap required, opened a new log
I20260926 14:59:44.460906 27873 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ef0a66ffec41ea9fe22c7a5d9b3532" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43431 } }
I20260926 14:59:44.461026 27873 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:44.461046 27873 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64ef0a66ffec41ea9fe22c7a5d9b3532, State: Initialized, Role: FOLLOWER
I20260926 14:59:44.461162 27873 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ef0a66ffec41ea9fe22c7a5d9b3532" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43431 } }
I20260926 14:59:44.461211 27873 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:44.461230 27873 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:44.461258 27873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:44.461661 27873 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ef0a66ffec41ea9fe22c7a5d9b3532" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43431 } }
I20260926 14:59:44.461743 27873 leader_election.cc:302] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64ef0a66ffec41ea9fe22c7a5d9b3532; no voters:
I20260926 14:59:44.461851 27873 leader_election.cc:288] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:44.461903 27876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:44.462024 27876 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532 [term 1 LEADER]: Becoming Leader. State: Replica: 64ef0a66ffec41ea9fe22c7a5d9b3532, State: Running, Role: LEADER
I20260926 14:59:44.462090 27873 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:44.462097 27876 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ef0a66ffec41ea9fe22c7a5d9b3532" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43431 } }
I20260926 14:59:44.462481 27878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64ef0a66ffec41ea9fe22c7a5d9b3532. Latest consensus state: current_term: 1 leader_uuid: "64ef0a66ffec41ea9fe22c7a5d9b3532" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ef0a66ffec41ea9fe22c7a5d9b3532" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43431 } } }
I20260926 14:59:44.462566 27877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64ef0a66ffec41ea9fe22c7a5d9b3532" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ef0a66ffec41ea9fe22c7a5d9b3532" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43431 } } }
I20260926 14:59:44.462728 27877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:44.462630 27878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:44.463089 27886 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:44.463529 27886 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:44.464946 27886 catalog_manager.cc:1347] Generated new cluster ID: 5b0f9edc537749baad541ac2f8f96f02
I20260926 14:59:44.464998 27886 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:59:44.465095 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 27805
I20260926 14:59:44.465214 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:44.465378 7319 external_mini_cluster.cc:1428] Setting key f48881d3c3e42adac7b72161740e5064
Sep 26 14:59:44 dist-test-slave-f9xb krb5kdc[27792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 14:59:44.481673 27896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:44.465764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37457 (local address 127.0.0.1:43431)
0926 14:59:44.471855 (+ 6091us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:44.471863 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:44.471895 (+ 32us) server_negotiation.cc:407] Connection header received
0926 14:59:44.472000 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:44.472006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:44.472063 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:44.472197 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:44.472874 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:44.474280 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:44.475240 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:44.475863 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:44.475958 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:44.478404 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:44.478408 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:44.478409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:44.478419 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:44.480180 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:44.480529 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:44.480532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:44.480532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:44.480615 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:44.480790 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:44.480791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:44.480791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:44.481036 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:44.481166 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:44.481263 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 14:59:44.481375 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6049,"thread_start_us":69,"threads_started":1}
I20260926 14:59:44.482411 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:59:44.485570 27886 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:44.486004 27886 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:44.500141 27886 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64ef0a66ffec41ea9fe22c7a5d9b3532: Generated new TSK 0
I20260926 14:59:44.500300 27886 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:44.502219 27896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:44.495962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49329 (local address 127.0.0.1:43431)
0926 14:59:44.496008 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:44.496009 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:44.496114 (+ 105us) server_negotiation.cc:407] Connection header received
0926 14:59:44.496173 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:44.496173 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:44.496183 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:44.496236 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:44.496824 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:44.497523 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:44.498517 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:44.498804 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:44.498854 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:44.500087 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:44.500088 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:44.500089 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:44.500095 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:44.501343 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:44.501678 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:44.501678 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:44.501679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:44.501718 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:44.501869 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:44.501870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:44.501870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:44.501946 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:44.502005 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:44.502117 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 14:59:44.502169 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 14:59:44.504041 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 27805
2026-09-26T14:59:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1217 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:44 dist-test-slave-f9xb krb5kdc[27904](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:44 dist-test-slave-f9xb krb5kdc[27904](info): set up 2 sockets
Sep 26 14:59:44 dist-test-slave-f9xb krb5kdc[27904](info): commencing operation
krb5kdc: starting...
W20260926 14:59:45.439328 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.919s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:45 dist-test-slave-f9xb krb5kdc[27904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 14:59:45.539690 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44743
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:45.613863 27917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:45.614032 27917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:45.614060 27917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:45.615128 27917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:45.615165 27917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:45.615185 27917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:45.615199 27917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:45.615216 27917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:45.616611 27917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:45.616813 27917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:45.616984 27917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:45.621104 27923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:45.621212 27917 server_base.cc:1034] running on GCE node
W20260926 14:59:45.621114 27922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:45.621120 27925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:45.621519 27917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:45.621769 27917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:45.622897 27917 hybrid_clock.cc:648] HybridClock initialized: now 1790434785622885 us; error 28 us; skew 500 ppm
Sep 26 14:59:45 dist-test-slave-f9xb krb5kdc[27904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434785, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:45.624997 27917 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 14:59:45.625360 27917 webserver.cc:460] Webserver started at http://127.0.0.1:38837/ using document root <none> and password file <none>
I20260926 14:59:45.625531 27917 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:45.625567 27917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:45.625649 27917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:45.626494 27917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "e28753cc3d77496c95fca6e045c8ba7e"
format_stamp: "Formatted at 2026-09-26 14:59:45 on dist-test-slave-f9xb"
server_key: "e39ee57feae5c49f9a2fc1767972dcc8"
server_key_iv: "5e9c24fadb550b9260fdd0a16b65cdbe"
server_key_version: "encryptionkey@0"
I20260926 14:59:45.626746 27917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "e28753cc3d77496c95fca6e045c8ba7e"
format_stamp: "Formatted at 2026-09-26 14:59:45 on dist-test-slave-f9xb"
server_key: "e39ee57feae5c49f9a2fc1767972dcc8"
server_key_iv: "5e9c24fadb550b9260fdd0a16b65cdbe"
server_key_version: "encryptionkey@0"
I20260926 14:59:45.627802 27917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:45.628382 27932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:45.628515 27917 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:45.628589 27917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "e28753cc3d77496c95fca6e045c8ba7e"
format_stamp: "Formatted at 2026-09-26 14:59:45 on dist-test-slave-f9xb"
server_key: "e39ee57feae5c49f9a2fc1767972dcc8"
server_key_iv: "5e9c24fadb550b9260fdd0a16b65cdbe"
server_key_version: "encryptionkey@0"
I20260926 14:59:45.628638 27917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:45.638507 27917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:45.640277 27917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:45.640372 27917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:45.643631 27917 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44743
I20260926 14:59:45.643659 27984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44743 every 8 connection(s)
I20260926 14:59:45.643950 27917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:45.644755 27985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:45.646592 27985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e: Bootstrap starting.
I20260926 14:59:45.647104 27985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:45.647349 27985 log.cc:824] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:45.647954 27985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e: No bootstrap required, opened a new log
I20260926 14:59:45.649379 27985 raft_consensus.cc:348] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e28753cc3d77496c95fca6e045c8ba7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44743 } }
I20260926 14:59:45.649494 27985 raft_consensus.cc:374] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:45.649523 27985 raft_consensus.cc:729] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e28753cc3d77496c95fca6e045c8ba7e, State: Initialized, Role: FOLLOWER
I20260926 14:59:45.649637 27985 consensus_queue.cc:260] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e28753cc3d77496c95fca6e045c8ba7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44743 } }
I20260926 14:59:45.649694 27985 raft_consensus.cc:388] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:45.649722 27985 raft_consensus.cc:482] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:45.649766 27985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:45.650208 27985 raft_consensus.cc:504] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e28753cc3d77496c95fca6e045c8ba7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44743 } }
I20260926 14:59:45.650300 27985 leader_election.cc:302] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e28753cc3d77496c95fca6e045c8ba7e; no voters:
I20260926 14:59:45.650442 27985 leader_election.cc:288] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:45.650470 27988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:45.650588 27988 raft_consensus.cc:686] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e [term 1 LEADER]: Becoming Leader. State: Replica: e28753cc3d77496c95fca6e045c8ba7e, State: Running, Role: LEADER
I20260926 14:59:45.650656 27985 sys_catalog.cc:564] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:45.650698 27988 consensus_queue.cc:237] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e28753cc3d77496c95fca6e045c8ba7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44743 } }
I20260926 14:59:45.651031 27990 sys_catalog.cc:455] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e28753cc3d77496c95fca6e045c8ba7e. Latest consensus state: current_term: 1 leader_uuid: "e28753cc3d77496c95fca6e045c8ba7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e28753cc3d77496c95fca6e045c8ba7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44743 } } }
I20260926 14:59:45.651127 27989 sys_catalog.cc:455] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e28753cc3d77496c95fca6e045c8ba7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e28753cc3d77496c95fca6e045c8ba7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44743 } } }
I20260926 14:59:45.651207 27989 sys_catalog.cc:458] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:45.651156 27990 sys_catalog.cc:458] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:45.651530 27997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:45.651969 27997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:45.653003 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 27917
I20260926 14:59:45.653108 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:45.653265 7319 external_mini_cluster.cc:1428] Setting key c9b4cf55c0cfeeb5b005eb5c5358f6e2
I20260926 14:59:45.653342 27997 catalog_manager.cc:1347] Generated new cluster ID: 6901e9bb11c745849c29dfb95824404a
I20260926 14:59:45.653374 27997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:59:45.658768 28008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:45.653623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59235 (local address 127.0.0.1:44743)
0926 14:59:45.653763 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:45.653771 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:45.653962 (+ 191us) server_negotiation.cc:407] Connection header received
0926 14:59:45.654081 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:45.654087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:45.654140 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:45.654291 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:45.654434 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:45.655855 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:45.656870 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:45.657465 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:45.657570 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:45.658125 (+ 555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:45.658129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:45.658129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:45.658324 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:45.658395 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:45.658411 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:45.658477 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:45.658529 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 14:59:45.658586 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
I20260926 14:59:45.659618 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:59:45.663939 28008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:45.660906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53675 (local address 127.0.0.1:44743)
0926 14:59:45.660935 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:45.660936 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:45.661047 (+ 111us) server_negotiation.cc:407] Connection header received
0926 14:59:45.661074 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:45.661075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:45.661084 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:45.661143 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:45.661234 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:45.661929 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:45.662825 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:45.663131 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:45.663184 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:45.663703 (+ 519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:45.663704 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:45.663705 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:45.663709 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:45.663722 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:45.663725 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:45.663804 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:45.663853 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 14:59:45.663872 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260926 14:59:45.682693 27997 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:45.683346 27997 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:45.688077 27997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e28753cc3d77496c95fca6e045c8ba7e: Generated new TSK 0
I20260926 14:59:45.688210 27997 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:45.696805 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 27917
2026-09-26T14:59:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1192 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:45 dist-test-slave-f9xb krb5kdc[28016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:45 dist-test-slave-f9xb krb5kdc[28016](info): set up 2 sockets
Sep 26 14:59:45 dist-test-slave-f9xb krb5kdc[28016](info): commencing operation
krb5kdc: starting...
W20260926 14:59:46.631472 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.918s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:46 dist-test-slave-f9xb krb5kdc[28016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 14:59:46.736466 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43415
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:46.812454 28029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:46.812623 28029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:46.812642 28029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:46.813681 28029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:46.813714 28029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:46.813726 28029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:46.813737 28029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:46.813750 28029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:46.815081 28029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:46.815284 28029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:46.815435 28029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:46.819597 28035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:46.819628 28029 server_base.cc:1034] running on GCE node
W20260926 14:59:46.819607 28034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:46.819585 28037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:46.820143 28029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:46.820391 28029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:46.821521 28029 hybrid_clock.cc:648] HybridClock initialized: now 1790434786821507 us; error 30 us; skew 500 ppm
Sep 26 14:59:46 dist-test-slave-f9xb krb5kdc[28016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434786, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:46.823580 28029 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 14:59:46.823951 28029 webserver.cc:460] Webserver started at http://127.0.0.1:35003/ using document root <none> and password file <none>
I20260926 14:59:46.824116 28029 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:46.824151 28029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:46.824231 28029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:46.825079 28029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "0f3df4cb42c246e6a45339cfba96138d"
format_stamp: "Formatted at 2026-09-26 14:59:46 on dist-test-slave-f9xb"
server_key: "17ac709adc073f2aa9e4838a41da1140"
server_key_iv: "2c9943fd53911200a95b8f7f08111599"
server_key_version: "encryptionkey@0"
I20260926 14:59:46.825340 28029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "0f3df4cb42c246e6a45339cfba96138d"
format_stamp: "Formatted at 2026-09-26 14:59:46 on dist-test-slave-f9xb"
server_key: "17ac709adc073f2aa9e4838a41da1140"
server_key_iv: "2c9943fd53911200a95b8f7f08111599"
server_key_version: "encryptionkey@0"
I20260926 14:59:46.826387 28029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:46.826937 28044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:46.827086 28029 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:46.827162 28029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "0f3df4cb42c246e6a45339cfba96138d"
format_stamp: "Formatted at 2026-09-26 14:59:46 on dist-test-slave-f9xb"
server_key: "17ac709adc073f2aa9e4838a41da1140"
server_key_iv: "2c9943fd53911200a95b8f7f08111599"
server_key_version: "encryptionkey@0"
I20260926 14:59:46.827234 28029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:46.830152 28029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:46.831653 28029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:46.831768 28029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:46.835021 28029 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43415
I20260926 14:59:46.835057 28096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43415 every 8 connection(s)
I20260926 14:59:46.835309 28029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:46.836161 28097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:46.837944 28097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d: Bootstrap starting.
I20260926 14:59:46.838380 28097 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:46.838584 28097 log.cc:824] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:46.839124 28097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d: No bootstrap required, opened a new log
I20260926 14:59:46.839726 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 28029
I20260926 14:59:46.839856 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:46.840011 7319 external_mini_cluster.cc:1428] Setting key 3d865ab0f62d150083cea9a06bf03b6a
I20260926 14:59:46.840958 28097 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3df4cb42c246e6a45339cfba96138d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43415 } }
I20260926 14:59:46.841104 28097 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:46.841138 28097 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f3df4cb42c246e6a45339cfba96138d, State: Initialized, Role: FOLLOWER
I20260926 14:59:46.841271 28097 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3df4cb42c246e6a45339cfba96138d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43415 } }
I20260926 14:59:46.841359 28097 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:46.841398 28097 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:46.841440 28097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:46.841984 28101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:46.840448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43293 (local address 127.0.0.1:43415)
0926 14:59:46.840617 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:46.840625 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:46.840655 (+ 30us) server_negotiation.cc:407] Connection header received
0926 14:59:46.840930 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:46.840937 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:46.840967 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:46.841099 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:46.841631 (+ 532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:46.841634 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:46.841634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:46.841637 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:46.841672 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:46.841683 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:46.841733 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:46.841812 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 14:59:46.841839 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
I20260926 14:59:46.842383 28097 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3df4cb42c246e6a45339cfba96138d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43415 } }
I20260926 14:59:46.842478 28097 leader_election.cc:302] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f3df4cb42c246e6a45339cfba96138d; no voters:
I20260926 14:59:46.842624 28097 leader_election.cc:288] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:46.842664 28102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:46.842800 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:59:46.842814 28102 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d [term 1 LEADER]: Becoming Leader. State: Replica: 0f3df4cb42c246e6a45339cfba96138d, State: Running, Role: LEADER
I20260926 14:59:46.842859 28097 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:46.842886 28102 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3df4cb42c246e6a45339cfba96138d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43415 } }
I20260926 14:59:46.843181 28103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f3df4cb42c246e6a45339cfba96138d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3df4cb42c246e6a45339cfba96138d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43415 } } }
I20260926 14:59:46.843245 28103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:46.843362 28103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f3df4cb42c246e6a45339cfba96138d. Latest consensus state: current_term: 1 leader_uuid: "0f3df4cb42c246e6a45339cfba96138d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3df4cb42c246e6a45339cfba96138d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43415 } } }
I20260926 14:59:46.843405 28103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d [sys.catalog]: This master's current role is: LEADER
W20260926 14:59:46.844396 28122 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 14:59:46.844446 28122 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 14:59:46.844594 28109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:46.844754 28109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:46.845479 28101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:46.844353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50827 (local address 127.0.0.1:43415)
0926 14:59:46.844381 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:46.844383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:46.844587 (+ 204us) server_negotiation.cc:407] Connection header received
0926 14:59:46.844629 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:46.844630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:46.844634 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:46.844690 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:46.845276 (+ 586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:46.845277 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:46.845277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:46.845281 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:46.845292 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:46.845296 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:46.845322 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:46.845417 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 14:59:46.845430 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260926 14:59:46.846079 28109 catalog_manager.cc:1347] Generated new cluster ID: 19966b5541494b1ebc2c755587437d7a
I20260926 14:59:46.846124 28109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:59:46.849418 28109 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:46.849570 28109 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:46.859305 28109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f3df4cb42c246e6a45339cfba96138d: Generated new TSK 0
I20260926 14:59:46.859442 28109 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:46.865708 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 28029
2026-09-26T14:59:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1168 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:46 dist-test-slave-f9xb krb5kdc[28128](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:46 dist-test-slave-f9xb krb5kdc[28128](info): set up 2 sockets
Sep 26 14:59:46 dist-test-slave-f9xb krb5kdc[28128](info): commencing operation
krb5kdc: starting...
W20260926 14:59:47.803448 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.922s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:47 dist-test-slave-f9xb krb5kdc[28128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 14:59:47.905980 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45491
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:47.981521 28141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:47.981719 28141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:47.981739 28141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:47.982843 28141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:47.982882 28141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:47.982893 28141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:47.982904 28141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:47.982914 28141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:47.984316 28141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:47.984506 28141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:47.984699 28141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:47.988961 28146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:47.988968 28149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:47.988969 28147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:47.989070 28141 server_base.cc:1034] running on GCE node
I20260926 14:59:47.989404 28141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:47.989650 28141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:47.990789 28141 hybrid_clock.cc:648] HybridClock initialized: now 1790434787990775 us; error 27 us; skew 500 ppm
Sep 26 14:59:47 dist-test-slave-f9xb krb5kdc[28128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434787, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:47.993003 28141 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 14:59:47.993371 28141 webserver.cc:460] Webserver started at http://127.0.0.1:34201/ using document root <none> and password file <none>
I20260926 14:59:47.993561 28141 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:47.993606 28141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:47.993726 28141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:47.994624 28141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "deefd3faf9144beeb0087a2b9936be78"
format_stamp: "Formatted at 2026-09-26 14:59:47 on dist-test-slave-f9xb"
server_key: "d760026c1076cb957c5039544fb7bcc0"
server_key_iv: "9a9a1d8d3422a42bcb8102872fd52016"
server_key_version: "encryptionkey@0"
I20260926 14:59:47.994937 28141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "deefd3faf9144beeb0087a2b9936be78"
format_stamp: "Formatted at 2026-09-26 14:59:47 on dist-test-slave-f9xb"
server_key: "d760026c1076cb957c5039544fb7bcc0"
server_key_iv: "9a9a1d8d3422a42bcb8102872fd52016"
server_key_version: "encryptionkey@0"
I20260926 14:59:47.996111 28141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 14:59:47.996716 28156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:47.996850 28141 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:47.996924 28141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "deefd3faf9144beeb0087a2b9936be78"
format_stamp: "Formatted at 2026-09-26 14:59:47 on dist-test-slave-f9xb"
server_key: "d760026c1076cb957c5039544fb7bcc0"
server_key_iv: "9a9a1d8d3422a42bcb8102872fd52016"
server_key_version: "encryptionkey@0"
I20260926 14:59:47.996984 28141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:48.008472 28141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:48.010075 28141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:48.010195 28141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:48.013610 28141 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45491
I20260926 14:59:48.013643 28208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45491 every 8 connection(s)
I20260926 14:59:48.013912 28141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:48.014706 28209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:48.016696 28209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78: Bootstrap starting.
I20260926 14:59:48.017190 28209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:48.017419 28209 log.cc:824] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:48.018065 28209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78: No bootstrap required, opened a new log
I20260926 14:59:48.019364 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 28141
I20260926 14:59:48.019480 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:48.019671 7319 external_mini_cluster.cc:1428] Setting key fd4a28463a5ce1bf567a137e659d96ea
I20260926 14:59:48.019793 28209 raft_consensus.cc:348] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deefd3faf9144beeb0087a2b9936be78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45491 } }
I20260926 14:59:48.019929 28209 raft_consensus.cc:374] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:48.019954 28209 raft_consensus.cc:729] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: deefd3faf9144beeb0087a2b9936be78, State: Initialized, Role: FOLLOWER
I20260926 14:59:48.020319 28209 consensus_queue.cc:260] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deefd3faf9144beeb0087a2b9936be78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45491 } }
I20260926 14:59:48.020401 28209 raft_consensus.cc:388] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:48.020424 28209 raft_consensus.cc:482] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:48.020469 28209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:48.021440 28209 raft_consensus.cc:504] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deefd3faf9144beeb0087a2b9936be78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45491 } }
I20260926 14:59:48.021523 28209 leader_election.cc:302] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: deefd3faf9144beeb0087a2b9936be78; no voters:
I20260926 14:59:48.021672 28209 leader_election.cc:288] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:48.021713 28214 raft_consensus.cc:2781] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:48.021806 28214 raft_consensus.cc:686] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78 [term 1 LEADER]: Becoming Leader. State: Replica: deefd3faf9144beeb0087a2b9936be78, State: Running, Role: LEADER
I20260926 14:59:48.021914 28209 sys_catalog.cc:564] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:48.021935 28214 consensus_queue.cc:237] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deefd3faf9144beeb0087a2b9936be78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45491 } }
I20260926 14:59:48.022246 28214 sys_catalog.cc:455] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader deefd3faf9144beeb0087a2b9936be78. Latest consensus state: current_term: 1 leader_uuid: "deefd3faf9144beeb0087a2b9936be78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deefd3faf9144beeb0087a2b9936be78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45491 } } }
I20260926 14:59:48.022470 28214 sys_catalog.cc:458] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:48.022390 28215 sys_catalog.cc:455] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "deefd3faf9144beeb0087a2b9936be78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deefd3faf9144beeb0087a2b9936be78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45491 } } }
I20260926 14:59:48.022845 28215 sys_catalog.cc:458] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:48.023033 28223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:48.023495 28223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:48.025260 28223 catalog_manager.cc:1347] Generated new cluster ID: 404dd6d524e641a997f13ebf3159da88
I20260926 14:59:48.025305 28223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 14:59:48 dist-test-slave-f9xb krb5kdc[28128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 14:59:48.029989 28213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:48.020103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:34577 (local address 127.0.0.1:45491)
0926 14:59:48.020274 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:48.020282 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:48.020338 (+ 56us) server_negotiation.cc:407] Connection header received
0926 14:59:48.020442 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:48.020448 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:48.020502 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:48.020619 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:48.021347 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:48.022508 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:48.023572 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:48.024063 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:48.024171 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:48.026431 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:48.026434 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:48.026434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:48.026443 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:48.027965 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:48.028405 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:48.028408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:48.028408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:48.028521 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:48.028765 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:48.028766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:48.028766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:48.029048 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:48.029595 (+ 547us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:48.029676 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 14:59:48.029783 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260926 14:59:48.030939 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:59:48.040915 28213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:48.032317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:44813 (local address 127.0.0.1:45491)
0926 14:59:48.035829 (+ 3512us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:48.035830 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:48.035838 (+ 8us) server_negotiation.cc:407] Connection header received
0926 14:59:48.035852 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:48.035853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:48.035861 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:48.035908 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:48.036361 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:48.036832 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:48.037473 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:48.037688 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:48.037739 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:48.038747 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:48.038749 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:48.038749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:48.038755 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:48.039943 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:48.040261 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:48.040262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:48.040262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:48.040304 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:48.040479 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:48.040479 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:48.040480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:48.040570 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:48.040654 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:48.040783 (+ 129us) server_negotiation.cc:299] Negotiation successful
0926 14:59:48.040836 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3491}
I20260926 14:59:48.043972 28223 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:48.044482 28223 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:48.055054 28223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P deefd3faf9144beeb0087a2b9936be78: Generated new TSK 0
I20260926 14:59:48.055233 28223 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:48.061859 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 28141
W20260926 14:59:48.065359 28126 connection.cc:535] client connection to 127.0.0.1:45491 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45491): Connection reset by peer (error 104)
2026-09-26T14:59:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1196 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:48 dist-test-slave-f9xb krb5kdc[28240](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:48 dist-test-slave-f9xb krb5kdc[28240](info): set up 2 sockets
Sep 26 14:59:48 dist-test-slave-f9xb krb5kdc[28240](info): commencing operation
krb5kdc: starting...
W20260926 14:59:48.995376 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.917s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:49 dist-test-slave-f9xb krb5kdc[28240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 14:59:49.102509 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45097
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:49.177090 28253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:49.177276 28253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:49.177305 28253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:49.178398 28253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:49.178435 28253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:49.178453 28253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:49.178468 28253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:49.178484 28253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:49.179895 28253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:49.180110 28253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:49.180301 28253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:49.184509 28258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:49.184523 28261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:49.184551 28259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:49.184695 28253 server_base.cc:1034] running on GCE node
I20260926 14:59:49.184952 28253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:49.185214 28253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:49.186352 28253 hybrid_clock.cc:648] HybridClock initialized: now 1790434789186337 us; error 30 us; skew 500 ppm
Sep 26 14:59:49 dist-test-slave-f9xb krb5kdc[28240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434789, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:49.188632 28253 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 14:59:49.189000 28253 webserver.cc:460] Webserver started at http://127.0.0.1:44619/ using document root <none> and password file <none>
I20260926 14:59:49.189170 28253 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:49.189216 28253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:49.189301 28253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:49.190198 28253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "cf0efed0c4204785a46403be38559304"
format_stamp: "Formatted at 2026-09-26 14:59:49 on dist-test-slave-f9xb"
server_key: "4919f9e6ecf17b36892e1a056d694fb2"
server_key_iv: "b2b33e24a64d2cc42acb85d3a32e660b"
server_key_version: "encryptionkey@0"
I20260926 14:59:49.190476 28253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "cf0efed0c4204785a46403be38559304"
format_stamp: "Formatted at 2026-09-26 14:59:49 on dist-test-slave-f9xb"
server_key: "4919f9e6ecf17b36892e1a056d694fb2"
server_key_iv: "b2b33e24a64d2cc42acb85d3a32e660b"
server_key_version: "encryptionkey@0"
I20260926 14:59:49.191536 28253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 14:59:49.192214 28268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:49.192373 28253 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:49.192451 28253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "cf0efed0c4204785a46403be38559304"
format_stamp: "Formatted at 2026-09-26 14:59:49 on dist-test-slave-f9xb"
server_key: "4919f9e6ecf17b36892e1a056d694fb2"
server_key_iv: "b2b33e24a64d2cc42acb85d3a32e660b"
server_key_version: "encryptionkey@0"
I20260926 14:59:49.192507 28253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:49.208909 28253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:49.210777 28253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:49.210897 28253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:49.214365 28253 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45097
I20260926 14:59:49.214394 28320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45097 every 8 connection(s)
I20260926 14:59:49.214663 28253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:49.215530 28321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:49.216105 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 28253
I20260926 14:59:49.216195 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:49.216356 7319 external_mini_cluster.cc:1428] Setting key 6333d3ccc6db511ca304302f47436598
I20260926 14:59:49.218165 28321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304: Bootstrap starting.
I20260926 14:59:49.218772 28321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:49.219027 28321 log.cc:824] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:49.219718 28321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304: No bootstrap required, opened a new log
I20260926 14:59:49.221726 28324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:49.216769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:39363 (local address 127.0.0.1:45097)
0926 14:59:49.216922 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:49.216931 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:49.216960 (+ 29us) server_negotiation.cc:407] Connection header received
0926 14:59:49.217405 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:49.217413 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:49.217473 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:49.217610 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:49.217773 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:49.219058 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:49.220087 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:49.220488 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:49.220570 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:49.221156 (+ 586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:49.221160 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:49.221161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:49.221322 (+ 161us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:49.221352 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:49.221362 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:49.221425 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:49.221524 (+ 99us) server_negotiation.cc:299] Negotiation successful
0926 14:59:49.221550 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
I20260926 14:59:49.221688 28321 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf0efed0c4204785a46403be38559304" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45097 } }
I20260926 14:59:49.221797 28321 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:49.221827 28321 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf0efed0c4204785a46403be38559304, State: Initialized, Role: FOLLOWER
I20260926 14:59:49.221947 28321 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf0efed0c4204785a46403be38559304" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45097 } }
I20260926 14:59:49.222019 28321 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:49.222040 28321 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:49.222083 28321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:49.222596 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:59:49.222620 28321 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf0efed0c4204785a46403be38559304" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45097 } }
I20260926 14:59:49.222698 28321 leader_election.cc:302] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf0efed0c4204785a46403be38559304; no voters:
I20260926 14:59:49.222843 28321 leader_election.cc:288] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:49.222869 28326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:49.223017 28326 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304 [term 1 LEADER]: Becoming Leader. State: Replica: cf0efed0c4204785a46403be38559304, State: Running, Role: LEADER
I20260926 14:59:49.223055 28321 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:49.223759 28326 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf0efed0c4204785a46403be38559304" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45097 } }
I20260926 14:59:49.224572 28327 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf0efed0c4204785a46403be38559304. Latest consensus state: current_term: 1 leader_uuid: "cf0efed0c4204785a46403be38559304" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf0efed0c4204785a46403be38559304" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45097 } } }
I20260926 14:59:49.224647 28327 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304 [sys.catalog]: This master's current role is: LEADER
W20260926 14:59:49.224866 28345 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 14:59:49.224936 28345 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 14:59:49.224927 28328 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf0efed0c4204785a46403be38559304" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf0efed0c4204785a46403be38559304" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45097 } } }
I20260926 14:59:49.225016 28328 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:49.225142 28347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:49.225306 28347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:49.226621 28347 catalog_manager.cc:1347] Generated new cluster ID: 40e9778547244914b516510850855be1
I20260926 14:59:49.226665 28347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:59:49.227226 28324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:49.224052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:59433 (local address 127.0.0.1:45097)
0926 14:59:49.224082 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:49.224084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:49.224090 (+ 6us) server_negotiation.cc:407] Connection header received
0926 14:59:49.224107 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:49.224108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:49.224116 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:49.224159 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:49.224341 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:49.224995 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:49.226002 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:49.226281 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:49.226348 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:49.226922 (+ 574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:49.226923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:49.226923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:49.226927 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:49.226939 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:49.226943 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:49.226994 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:49.227078 (+ 84us) server_negotiation.cc:299] Negotiation successful
0926 14:59:49.227086 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 14:59:49.246426 28347 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:49.246918 28347 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:49.249485 28347 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf0efed0c4204785a46403be38559304: Generated new TSK 0
I20260926 14:59:49.249634 28347 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:49.257170 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 28253
W20260926 14:59:49.262269 28238 connection.cc:535] client connection to 127.0.0.1:45097 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45097): Connection reset by peer (error 104)
2026-09-26T14:59:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1196 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:49 dist-test-slave-f9xb krb5kdc[28352](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:49 dist-test-slave-f9xb krb5kdc[28352](info): set up 2 sockets
Sep 26 14:59:49 dist-test-slave-f9xb krb5kdc[28352](info): commencing operation
krb5kdc: starting...
W20260926 14:59:50.191416 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.916s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:50 dist-test-slave-f9xb krb5kdc[28352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 14:59:50.291608 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35833
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:50.366391 28365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:50.366562 28365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:50.366592 28365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:50.367681 28365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:50.367719 28365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:50.367756 28365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:50.367775 28365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:50.367791 28365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:50.369192 28365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:50.369393 28365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:50.369577 28365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:50.373895 28370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:50.373932 28373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:50.373903 28371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:50.374095 28365 server_base.cc:1034] running on GCE node
I20260926 14:59:50.374289 28365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:50.374534 28365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:50.375681 28365 hybrid_clock.cc:648] HybridClock initialized: now 1790434790375665 us; error 31 us; skew 500 ppm
Sep 26 14:59:50 dist-test-slave-f9xb krb5kdc[28352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434790, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:50.381434 28365 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 14:59:50.381801 28365 webserver.cc:460] Webserver started at http://127.0.0.1:37275/ using document root <none> and password file <none>
I20260926 14:59:50.381991 28365 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:50.382037 28365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:50.382156 28365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:50.383046 28365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "ffd9eea921914add8cb1b34eb7d030c4"
format_stamp: "Formatted at 2026-09-26 14:59:50 on dist-test-slave-f9xb"
server_key: "a5ff668d017c1dab3e0bd336ee704cd1"
server_key_iv: "c859a58f9af0c43d9d9cec88e7e5bac7"
server_key_version: "encryptionkey@0"
I20260926 14:59:50.383337 28365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "ffd9eea921914add8cb1b34eb7d030c4"
format_stamp: "Formatted at 2026-09-26 14:59:50 on dist-test-slave-f9xb"
server_key: "a5ff668d017c1dab3e0bd336ee704cd1"
server_key_iv: "c859a58f9af0c43d9d9cec88e7e5bac7"
server_key_version: "encryptionkey@0"
I20260926 14:59:50.384523 28365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 14:59:50.385133 28380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:50.385274 28365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:50.385345 28365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "ffd9eea921914add8cb1b34eb7d030c4"
format_stamp: "Formatted at 2026-09-26 14:59:50 on dist-test-slave-f9xb"
server_key: "a5ff668d017c1dab3e0bd336ee704cd1"
server_key_iv: "c859a58f9af0c43d9d9cec88e7e5bac7"
server_key_version: "encryptionkey@0"
I20260926 14:59:50.385399 28365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:50.388350 28365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:50.389861 28365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:50.389973 28365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:50.393347 28365 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35833
I20260926 14:59:50.393386 28432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35833 every 8 connection(s)
I20260926 14:59:50.393630 28365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:50.394440 28433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:50.394949 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 28365
I20260926 14:59:50.395054 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:50.395205 7319 external_mini_cluster.cc:1428] Setting key 8fd54ca72b5637811421f91cc45a66fb
I20260926 14:59:50.396929 28433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4: Bootstrap starting.
I20260926 14:59:50.397203 28436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:50.395616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:38899 (local address 127.0.0.1:35833)
0926 14:59:50.395777 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:50.395785 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:50.395813 (+ 28us) server_negotiation.cc:407] Connection header received
0926 14:59:50.396167 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:50.396173 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:50.396195 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:50.396293 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:50.396841 (+ 548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:50.396844 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:50.396844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:50.396847 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:50.396880 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:50.396888 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:50.396937 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:50.397039 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 14:59:50.397073 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260926 14:59:50.397526 28433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:50.397804 28433 log.cc:824] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:50.397980 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:59:50.398397 28433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4: No bootstrap required, opened a new log
I20260926 14:59:50.400012 28436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:50.399158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:54641 (local address 127.0.0.1:35833)
0926 14:59:50.399191 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:50.399192 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:50.399258 (+ 66us) server_negotiation.cc:407] Connection header received
0926 14:59:50.399326 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:50.399327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:50.399331 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:50.399372 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:50.399841 (+ 469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:50.399842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:50.399842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:50.399846 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:50.399856 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:50.399860 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:50.399903 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:50.399949 (+ 46us) server_negotiation.cc:299] Negotiation successful
0926 14:59:50.399960 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 14:59:50.400327 28433 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd9eea921914add8cb1b34eb7d030c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35833 } }
I20260926 14:59:50.400472 28433 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:50.400509 28433 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffd9eea921914add8cb1b34eb7d030c4, State: Initialized, Role: FOLLOWER
I20260926 14:59:50.400588 28433 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd9eea921914add8cb1b34eb7d030c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35833 } }
I20260926 14:59:50.400642 28433 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:50.400661 28433 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:50.400692 28433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:50.401098 28433 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd9eea921914add8cb1b34eb7d030c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35833 } }
I20260926 14:59:50.401183 28433 leader_election.cc:302] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffd9eea921914add8cb1b34eb7d030c4; no voters:
I20260926 14:59:50.401297 28433 leader_election.cc:288] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:50.401367 28443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:50.401484 28433 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:50.401517 28443 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4 [term 1 LEADER]: Becoming Leader. State: Replica: ffd9eea921914add8cb1b34eb7d030c4, State: Running, Role: LEADER
I20260926 14:59:50.401594 28443 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd9eea921914add8cb1b34eb7d030c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35833 } }
I20260926 14:59:50.401929 28444 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffd9eea921914add8cb1b34eb7d030c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd9eea921914add8cb1b34eb7d030c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35833 } } }
I20260926 14:59:50.401935 28445 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffd9eea921914add8cb1b34eb7d030c4. Latest consensus state: current_term: 1 leader_uuid: "ffd9eea921914add8cb1b34eb7d030c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd9eea921914add8cb1b34eb7d030c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35833 } } }
I20260926 14:59:50.402076 28444 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:50.402087 28445 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:50.402603 28452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:50.403041 28452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:50.404440 28452 catalog_manager.cc:1347] Generated new cluster ID: ace703c66a0a402da47a6dd992f5ac8b
I20260926 14:59:50.404495 28452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:59:50.413870 28452 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:50.414007 28452 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:50.423980 28452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffd9eea921914add8cb1b34eb7d030c4: Generated new TSK 0
I20260926 14:59:50.424126 28452 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:50.429203 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 28365
2026-09-26T14:59:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1170 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:50 dist-test-slave-f9xb krb5kdc[28464](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:50 dist-test-slave-f9xb krb5kdc[28464](info): set up 2 sockets
Sep 26 14:59:50 dist-test-slave-f9xb krb5kdc[28464](info): commencing operation
krb5kdc: starting...
W20260926 14:59:51.367527 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.922s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:51 dist-test-slave-f9xb krb5kdc[28464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 14:59:51.469796 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34109
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:51.545161 28477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:51.545363 28477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:51.545387 28477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:51.546490 28477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:51.546528 28477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:51.546540 28477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:51.546564 28477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:51.546587 28477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:51.547976 28477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34109
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:51.548164 28477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:51.548352 28477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:51.552511 28485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:51.552568 28482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:51.552498 28483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:51.552712 28477 server_base.cc:1034] running on GCE node
I20260926 14:59:51.552909 28477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:51.553166 28477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:51.554307 28477 hybrid_clock.cc:648] HybridClock initialized: now 1790434791554296 us; error 28 us; skew 500 ppm
Sep 26 14:59:51 dist-test-slave-f9xb krb5kdc[28464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434791, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:51.556478 28477 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 14:59:51.556859 28477 webserver.cc:460] Webserver started at http://127.0.0.1:35337/ using document root <none> and password file <none>
I20260926 14:59:51.557053 28477 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:51.557099 28477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:51.557214 28477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:51.558099 28477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "114301e238c746d38347bfc068d689b6"
format_stamp: "Formatted at 2026-09-26 14:59:51 on dist-test-slave-f9xb"
server_key: "367fee6ed66aafd8a3f65706e0783be9"
server_key_iv: "8c7c450a8ebd065bd5186d9c8b66235f"
server_key_version: "encryptionkey@0"
I20260926 14:59:51.558389 28477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "114301e238c746d38347bfc068d689b6"
format_stamp: "Formatted at 2026-09-26 14:59:51 on dist-test-slave-f9xb"
server_key: "367fee6ed66aafd8a3f65706e0783be9"
server_key_iv: "8c7c450a8ebd065bd5186d9c8b66235f"
server_key_version: "encryptionkey@0"
I20260926 14:59:51.559499 28477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:51.560171 28492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:51.560316 28477 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:51.560393 28477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "114301e238c746d38347bfc068d689b6"
format_stamp: "Formatted at 2026-09-26 14:59:51 on dist-test-slave-f9xb"
server_key: "367fee6ed66aafd8a3f65706e0783be9"
server_key_iv: "8c7c450a8ebd065bd5186d9c8b66235f"
server_key_version: "encryptionkey@0"
I20260926 14:59:51.560456 28477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:51.574453 28477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:51.576000 28477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:51.576113 28477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:51.579471 28477 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34109
I20260926 14:59:51.579499 28544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34109 every 8 connection(s)
I20260926 14:59:51.579795 28477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:51.580602 28545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:51.582453 28545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6: Bootstrap starting.
I20260926 14:59:51.582898 28545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:51.583117 28545 log.cc:824] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:51.583550 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 28477
I20260926 14:59:51.583673 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:51.583760 28545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6: No bootstrap required, opened a new log
I20260926 14:59:51.583897 7319 external_mini_cluster.cc:1428] Setting key 1c55c444fc4085f289dc7d2cca5211c3
I20260926 14:59:51.585860 28545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "114301e238c746d38347bfc068d689b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34109 } }
I20260926 14:59:51.585981 28545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:51.586001 28545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 114301e238c746d38347bfc068d689b6, State: Initialized, Role: FOLLOWER
I20260926 14:59:51.586093 28545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "114301e238c746d38347bfc068d689b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34109 } }
I20260926 14:59:51.586166 28545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:51.586189 28545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:51.586230 28545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:51.586690 28545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "114301e238c746d38347bfc068d689b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34109 } }
I20260926 14:59:51.586768 28545 leader_election.cc:302] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 114301e238c746d38347bfc068d689b6; no voters:
I20260926 14:59:51.586906 28545 leader_election.cc:288] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:51.586926 28550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:51.587025 28550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6 [term 1 LEADER]: Becoming Leader. State: Replica: 114301e238c746d38347bfc068d689b6, State: Running, Role: LEADER
I20260926 14:59:51.587136 28545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:51.587153 28550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "114301e238c746d38347bfc068d689b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34109 } }
I20260926 14:59:51.587498 28550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 114301e238c746d38347bfc068d689b6. Latest consensus state: current_term: 1 leader_uuid: "114301e238c746d38347bfc068d689b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "114301e238c746d38347bfc068d689b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34109 } } }
I20260926 14:59:51.587513 28551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "114301e238c746d38347bfc068d689b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "114301e238c746d38347bfc068d689b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34109 } } }
I20260926 14:59:51.587579 28550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:51.587604 28551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6 [sys.catalog]: This master's current role is: LEADER
W20260926 14:59:51.588878 28565 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 14:59:51.589227 28565 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 14:59:51.589282 28566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:51.589427 28566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 14:59:51 dist-test-slave-f9xb krb5kdc[28464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 14:59:51.590862 28566 catalog_manager.cc:1347] Generated new cluster ID: 10a795d9673e47e8989039f19c1e6b05
I20260926 14:59:51.590909 28566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:59:51.594347 28549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:51.584348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:38647 (local address 127.0.0.1:34109)
0926 14:59:51.584490 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:51.584497 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:51.584526 (+ 29us) server_negotiation.cc:407] Connection header received
0926 14:59:51.584987 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:51.584993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:51.585036 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:51.585136 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:51.585820 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:51.586992 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:51.588003 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:51.588613 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:51.588717 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:51.591040 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:51.591044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:51.591044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:51.591054 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:51.592864 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:51.593243 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:51.593245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:51.593246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:51.593346 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:51.593540 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:51.593541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:51.593541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:51.593833 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:51.593989 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:51.594068 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 14:59:51.594158 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260926 14:59:51.595279 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:59:51.603785 28566 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:51.604338 28566 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:51.606447 28549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:51.600052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:47459 (local address 127.0.0.1:34109)
0926 14:59:51.600085 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:51.600087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:51.600141 (+ 54us) server_negotiation.cc:407] Connection header received
0926 14:59:51.600193 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:51.600194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:51.600202 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:51.600253 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:51.600763 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:51.601475 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:51.602366 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:51.602667 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:51.602739 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:51.603940 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:51.603941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:51.603942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:51.603949 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:51.605428 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:51.605791 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:51.605792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:51.605793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:51.605844 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:51.606017 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:51.606018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:51.606018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:51.606130 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:51.606220 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:51.606313 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 14:59:51.606370 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 14:59:51.617547 28566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 114301e238c746d38347bfc068d689b6: Generated new TSK 0
I20260926 14:59:51.617708 28566 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:51.620630 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 28477
W20260926 14:59:51.624205 28462 connection.cc:535] client connection to 127.0.0.1:34109 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34109): Connection reset by peer (error 104)
2026-09-26T14:59:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1191 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:51 dist-test-slave-f9xb krb5kdc[28576](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:51 dist-test-slave-f9xb krb5kdc[28576](info): set up 2 sockets
Sep 26 14:59:51 dist-test-slave-f9xb krb5kdc[28576](info): commencing operation
krb5kdc: starting...
W20260926 14:59:52.575404 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:52 dist-test-slave-f9xb krb5kdc[28576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 14:59:52.677680 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35125
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:52.752207 28589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:52.752381 28589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:52.752401 28589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:52.753496 28589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:52.753535 28589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:52.753547 28589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:52.753569 28589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:52.753599 28589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:52.754988 28589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35125
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:52.755191 28589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:52.755373 28589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:52.759534 28594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:52.759572 28597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:52.759626 28595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:52.759773 28589 server_base.cc:1034] running on GCE node
I20260926 14:59:52.759966 28589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:52.760224 28589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:52.761361 28589 hybrid_clock.cc:648] HybridClock initialized: now 1790434792761345 us; error 29 us; skew 500 ppm
Sep 26 14:59:52 dist-test-slave-f9xb krb5kdc[28576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434792, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:52.763432 28589 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 14:59:52.763829 28589 webserver.cc:460] Webserver started at http://127.0.0.1:39647/ using document root <none> and password file <none>
I20260926 14:59:52.764006 28589 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:52.764050 28589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:52.764165 28589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:52.765064 28589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "c9ed82938c384468b70d390c11cd2e5b"
format_stamp: "Formatted at 2026-09-26 14:59:52 on dist-test-slave-f9xb"
server_key: "6770fbbe50f631aadbb25a12aef20cbd"
server_key_iv: "8cf4bbf4c2fc4800e57220bd2b3ad1a0"
server_key_version: "encryptionkey@0"
I20260926 14:59:52.765358 28589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "c9ed82938c384468b70d390c11cd2e5b"
format_stamp: "Formatted at 2026-09-26 14:59:52 on dist-test-slave-f9xb"
server_key: "6770fbbe50f631aadbb25a12aef20cbd"
server_key_iv: "8cf4bbf4c2fc4800e57220bd2b3ad1a0"
server_key_version: "encryptionkey@0"
I20260926 14:59:52.766466 28589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:52.767115 28604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:52.767254 28589 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:52.767331 28589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "c9ed82938c384468b70d390c11cd2e5b"
format_stamp: "Formatted at 2026-09-26 14:59:52 on dist-test-slave-f9xb"
server_key: "6770fbbe50f631aadbb25a12aef20cbd"
server_key_iv: "8cf4bbf4c2fc4800e57220bd2b3ad1a0"
server_key_version: "encryptionkey@0"
I20260926 14:59:52.767395 28589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:52.776764 28589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:52.778298 28589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:52.778429 28589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:52.782147 28589 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35125
I20260926 14:59:52.782181 28656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35125 every 8 connection(s)
I20260926 14:59:52.782456 28589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:52.783221 28657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:52.785099 28657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b: Bootstrap starting.
I20260926 14:59:52.785602 28657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:52.785826 28657 log.cc:824] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:52.786397 28657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b: No bootstrap required, opened a new log
I20260926 14:59:52.787868 28657 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ed82938c384468b70d390c11cd2e5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35125 } }
I20260926 14:59:52.787984 28657 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:52.788004 28657 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ed82938c384468b70d390c11cd2e5b, State: Initialized, Role: FOLLOWER
I20260926 14:59:52.788123 28657 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ed82938c384468b70d390c11cd2e5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35125 } }
I20260926 14:59:52.788219 28657 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:52.788252 28657 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:52.788296 28657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:52.788735 28657 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ed82938c384468b70d390c11cd2e5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35125 } }
I20260926 14:59:52.788813 28657 leader_election.cc:302] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9ed82938c384468b70d390c11cd2e5b; no voters:
I20260926 14:59:52.788939 28657 leader_election.cc:288] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:52.788978 28660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:52.789134 28657 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:52.789144 28660 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b [term 1 LEADER]: Becoming Leader. State: Replica: c9ed82938c384468b70d390c11cd2e5b, State: Running, Role: LEADER
I20260926 14:59:52.789294 28660 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ed82938c384468b70d390c11cd2e5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35125 } }
I20260926 14:59:52.789731 28661 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9ed82938c384468b70d390c11cd2e5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ed82938c384468b70d390c11cd2e5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35125 } } }
I20260926 14:59:52.789829 28661 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:52.789978 28662 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9ed82938c384468b70d390c11cd2e5b. Latest consensus state: current_term: 1 leader_uuid: "c9ed82938c384468b70d390c11cd2e5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ed82938c384468b70d390c11cd2e5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35125 } } }
I20260926 14:59:52.790007 28669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:52.790040 28662 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:52.790495 28669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:52.790781 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 28589
I20260926 14:59:52.790884 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:52.791049 7319 external_mini_cluster.cc:1428] Setting key 4d5ad1947adc1b80f198703884d82697
I20260926 14:59:52.792490 28669 catalog_manager.cc:1347] Generated new cluster ID: 66b747e366674b52bec1882f30d00dc4
I20260926 14:59:52.792536 28669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:59:52.796404 28679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:52.791396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:41799 (local address 127.0.0.1:35125)
0926 14:59:52.791690 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:52.791699 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:52.791720 (+ 21us) server_negotiation.cc:407] Connection header received
0926 14:59:52.791834 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:52.791840 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:52.791883 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:52.792001 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:52.792184 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:52.793482 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:52.794501 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:52.795114 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:52.795221 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:52.795800 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:52.795804 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:52.795805 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:52.795986 (+ 181us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:52.796020 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:52.796034 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:52.796118 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:52.796191 (+ 73us) server_negotiation.cc:299] Negotiation successful
0926 14:59:52.796214 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":71,"threads_started":1}
I20260926 14:59:52.797425 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:59:52.802851 28669 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:52.803289 28669 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:52.805867 28679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:52.799989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:49069 (local address 127.0.0.1:35125)
0926 14:59:52.800025 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:52.800026 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:52.802668 (+ 2642us) server_negotiation.cc:407] Connection header received
0926 14:59:52.802723 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:52.802723 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:52.802732 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:52.802787 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:52.802950 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:52.803672 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:52.804716 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:52.805007 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:52.805066 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:52.805649 (+ 583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:52.805650 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:52.805650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:52.805660 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:52.805672 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:52.805676 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:52.805726 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:52.805794 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 14:59:52.805804 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 14:59:52.812896 28669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9ed82938c384468b70d390c11cd2e5b: Generated new TSK 0
I20260926 14:59:52.813027 28669 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:52.820784 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 28589
W20260926 14:59:52.824136 28574 connection.cc:535] client connection to 127.0.0.1:35125 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35125): Connection reset by peer (error 104)
2026-09-26T14:59:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1200 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:52 dist-test-slave-f9xb krb5kdc[28688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:52 dist-test-slave-f9xb krb5kdc[28688](info): set up 2 sockets
Sep 26 14:59:52 dist-test-slave-f9xb krb5kdc[28688](info): commencing operation
krb5kdc: starting...
W20260926 14:59:53.767395 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:53 dist-test-slave-f9xb krb5kdc[28688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 14:59:53.869709 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35323
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:53.944299 28701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:53.944470 28701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:53.944489 28701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:53.945536 28701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:53.945568 28701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:53.945580 28701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:53.945591 28701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:53.945601 28701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:53.946929 28701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35323
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:53.947091 28701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:53.947242 28701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:53.951431 28706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:53.951411 28707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:53.951419 28709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:53.951506 28701 server_base.cc:1034] running on GCE node
I20260926 14:59:53.951826 28701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:53.952050 28701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:53.953179 28701 hybrid_clock.cc:648] HybridClock initialized: now 1790434793953167 us; error 28 us; skew 500 ppm
Sep 26 14:59:53 dist-test-slave-f9xb krb5kdc[28688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434793, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:53.955247 28701 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 14:59:53.955593 28701 webserver.cc:460] Webserver started at http://127.0.0.1:42669/ using document root <none> and password file <none>
I20260926 14:59:53.955801 28701 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:53.955842 28701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:53.955950 28701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:53.956857 28701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "28fadc3b65d3443ab9e9e750e856312d"
format_stamp: "Formatted at 2026-09-26 14:59:53 on dist-test-slave-f9xb"
server_key: "152569abf486f25f04a485d14590834d"
server_key_iv: "8344ef034be017a565f593f0584537a4"
server_key_version: "encryptionkey@0"
I20260926 14:59:53.957144 28701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "28fadc3b65d3443ab9e9e750e856312d"
format_stamp: "Formatted at 2026-09-26 14:59:53 on dist-test-slave-f9xb"
server_key: "152569abf486f25f04a485d14590834d"
server_key_iv: "8344ef034be017a565f593f0584537a4"
server_key_version: "encryptionkey@0"
I20260926 14:59:53.958261 28701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:53.958818 28716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:53.958943 28701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:53.959016 28701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "28fadc3b65d3443ab9e9e750e856312d"
format_stamp: "Formatted at 2026-09-26 14:59:53 on dist-test-slave-f9xb"
server_key: "152569abf486f25f04a485d14590834d"
server_key_iv: "8344ef034be017a565f593f0584537a4"
server_key_version: "encryptionkey@0"
I20260926 14:59:53.959071 28701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:53.962020 28701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:53.963517 28701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:53.963630 28701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:53.967002 28701 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35323
I20260926 14:59:53.967037 28768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35323 every 8 connection(s)
I20260926 14:59:53.967296 28701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:53.968209 28769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:53.970213 28769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d: Bootstrap starting.
I20260926 14:59:53.970690 28769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:53.970913 28769 log.cc:824] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:53.971477 28769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d: No bootstrap required, opened a new log
I20260926 14:59:53.972800 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 28701
I20260926 14:59:53.972905 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:53.973045 7319 external_mini_cluster.cc:1428] Setting key 3f0f4381deacd8752e8eaffb6fbaa967
I20260926 14:59:53.973100 28769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fadc3b65d3443ab9e9e750e856312d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35323 } }
I20260926 14:59:53.973266 28769 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:53.973313 28769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28fadc3b65d3443ab9e9e750e856312d, State: Initialized, Role: FOLLOWER
I20260926 14:59:53.973457 28769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fadc3b65d3443ab9e9e750e856312d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35323 } }
I20260926 14:59:53.973543 28769 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:53.973626 28769 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:53.973683 28769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:53.974598 28769 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fadc3b65d3443ab9e9e750e856312d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35323 } }
I20260926 14:59:53.974706 28769 leader_election.cc:302] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28fadc3b65d3443ab9e9e750e856312d; no voters:
I20260926 14:59:53.974855 28769 leader_election.cc:288] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:53.974870 28774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:53.975157 28769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:53.975155 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:53.973652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:38811 (local address 127.0.0.1:35323)
0926 14:59:53.973807 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:53.973814 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:53.973835 (+ 21us) server_negotiation.cc:407] Connection header received
0926 14:59:53.973919 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:53.973924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:53.973955 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:53.974047 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:53.974682 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:53.974685 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:53.974686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:53.974691 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:53.974738 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:53.974752 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:53.974814 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:53.974922 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 14:59:53.974964 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
I20260926 14:59:53.975179 28774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d [term 1 LEADER]: Becoming Leader. State: Replica: 28fadc3b65d3443ab9e9e750e856312d, State: Running, Role: LEADER
I20260926 14:59:53.975507 28774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fadc3b65d3443ab9e9e750e856312d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35323 } }
I20260926 14:59:53.976058 28776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28fadc3b65d3443ab9e9e750e856312d. Latest consensus state: current_term: 1 leader_uuid: "28fadc3b65d3443ab9e9e750e856312d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fadc3b65d3443ab9e9e750e856312d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35323 } } }
I20260926 14:59:53.976145 28776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:53.976218 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:59:53.976353 28784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:53.976378 28775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28fadc3b65d3443ab9e9e750e856312d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fadc3b65d3443ab9e9e750e856312d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35323 } } }
I20260926 14:59:53.976441 28775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:53.976774 28784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:53.979043 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:53.977898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:44131 (local address 127.0.0.1:35323)
0926 14:59:53.977931 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:53.977933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:53.978034 (+ 101us) server_negotiation.cc:407] Connection header received
0926 14:59:53.978094 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:53.978095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:53.978100 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:53.978151 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:53.978824 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:53.978825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:53.978825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:53.978829 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:53.978841 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:53.978845 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:53.978871 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:53.978968 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 14:59:53.978980 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 14:59:53.979240 28784 catalog_manager.cc:1347] Generated new cluster ID: 6ca61df8a29f49049e7daa7afaed6674
I20260926 14:59:53.979283 28784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:59:53.982278 28784 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:53.982414 28784 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:53.987155 28784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28fadc3b65d3443ab9e9e750e856312d: Generated new TSK 0
I20260926 14:59:53.987294 28784 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:53.994634 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 28701
2026-09-26T14:59:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1173 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:54 dist-test-slave-f9xb krb5kdc[28800](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:54 dist-test-slave-f9xb krb5kdc[28800](info): set up 2 sockets
Sep 26 14:59:54 dist-test-slave-f9xb krb5kdc[28800](info): commencing operation
krb5kdc: starting...
W20260926 14:59:54.939429 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.928s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:55 dist-test-slave-f9xb krb5kdc[28800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:59:55.042691 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:40687
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:37099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:40687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:55.119688 28813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:55.119882 28813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:55.119911 28813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:55.121014 28813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:55.121052 28813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:55.121070 28813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:55.121088 28813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:55.121101 28813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:55.122493 28813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:37099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:40687
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:40687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:55.122712 28813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:55.122905 28813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:55.127156 28819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:55.127233 28818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:55.127249 28813 server_base.cc:1034] running on GCE node
W20260926 14:59:55.127166 28821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:55.127575 28813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:55.127866 28813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:55.129001 28813 hybrid_clock.cc:648] HybridClock initialized: now 1790434795128986 us; error 27 us; skew 500 ppm
Sep 26 14:59:55 dist-test-slave-f9xb krb5kdc[28800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434795, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:55.131206 28813 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:59:55.131559 28813 webserver.cc:460] Webserver started at http://127.7.37.254:44441/ using document root <none> and password file <none>
I20260926 14:59:55.131789 28813 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:55.131839 28813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:55.131954 28813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:55.132851 28813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "28a32057e6b5455d8a1fa19d270b8bdf"
format_stamp: "Formatted at 2026-09-26 14:59:55 on dist-test-slave-f9xb"
server_key: "f35f26050a5f8a4575dd76a3d753a6a6"
server_key_iv: "5de7461a733a042481977765cdcfff17"
server_key_version: "encryptionkey@0"
I20260926 14:59:55.133147 28813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "28a32057e6b5455d8a1fa19d270b8bdf"
format_stamp: "Formatted at 2026-09-26 14:59:55 on dist-test-slave-f9xb"
server_key: "f35f26050a5f8a4575dd76a3d753a6a6"
server_key_iv: "5de7461a733a042481977765cdcfff17"
server_key_version: "encryptionkey@0"
I20260926 14:59:55.134248 28813 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 14:59:55.134869 28828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:55.135020 28813 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:55.135099 28813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "28a32057e6b5455d8a1fa19d270b8bdf"
format_stamp: "Formatted at 2026-09-26 14:59:55 on dist-test-slave-f9xb"
server_key: "f35f26050a5f8a4575dd76a3d753a6a6"
server_key_iv: "5de7461a733a042481977765cdcfff17"
server_key_version: "encryptionkey@0"
I20260926 14:59:55.135170 28813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:55.151422 28813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:55.152968 28813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:55.153082 28813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:55.156479 28813 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:40687
I20260926 14:59:55.156509 28880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:40687 every 8 connection(s)
I20260926 14:59:55.156774 28813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:55.157474 28881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:55.159423 28881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf: Bootstrap starting.
I20260926 14:59:55.159960 28881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:55.160207 28881 log.cc:824] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:55.160774 28881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf: No bootstrap required, opened a new log
I20260926 14:59:55.162230 28881 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28a32057e6b5455d8a1fa19d270b8bdf" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 40687 } }
I20260926 14:59:55.162350 28881 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:55.162371 28881 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28a32057e6b5455d8a1fa19d270b8bdf, State: Initialized, Role: FOLLOWER
I20260926 14:59:55.162496 28881 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28a32057e6b5455d8a1fa19d270b8bdf" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 40687 } }
I20260926 14:59:55.162551 28881 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:55.162572 28881 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:55.162613 28881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:55.163331 28881 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28a32057e6b5455d8a1fa19d270b8bdf" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 40687 } }
I20260926 14:59:55.163412 28881 leader_election.cc:302] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28a32057e6b5455d8a1fa19d270b8bdf; no voters:
I20260926 14:59:55.163546 28881 leader_election.cc:288] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:55.163596 28884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:55.163779 28881 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:55.163789 28884 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf [term 1 LEADER]: Becoming Leader. State: Replica: 28a32057e6b5455d8a1fa19d270b8bdf, State: Running, Role: LEADER
I20260926 14:59:55.163890 28884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28a32057e6b5455d8a1fa19d270b8bdf" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 40687 } }
I20260926 14:59:55.164214 28886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28a32057e6b5455d8a1fa19d270b8bdf. Latest consensus state: current_term: 1 leader_uuid: "28a32057e6b5455d8a1fa19d270b8bdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28a32057e6b5455d8a1fa19d270b8bdf" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 40687 } } }
I20260926 14:59:55.164222 28885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28a32057e6b5455d8a1fa19d270b8bdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28a32057e6b5455d8a1fa19d270b8bdf" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 40687 } } }
I20260926 14:59:55.164350 28886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:55.164566 28885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:55.164762 28893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:55.165120 28893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:55.166190 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 28813
I20260926 14:59:55.166304 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:55.166482 7319 external_mini_cluster.cc:1428] Setting key d9750c2f2075a06f5ff75c89fd798c8c
I20260926 14:59:55.166533 28893 catalog_manager.cc:1347] Generated new cluster ID: c0e3d5eee4744a2a8621888815694a52
I20260926 14:59:55.166567 28893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 14:59:55 dist-test-slave-f9xb krb5kdc[28800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:59:55.177970 28904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:55.166871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57911 (local address 127.7.37.254:40687)
0926 14:59:55.168049 (+ 1178us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:55.168058 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:55.168087 (+ 29us) server_negotiation.cc:407] Connection header received
0926 14:59:55.168200 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:55.168206 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:55.168256 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:55.168390 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:55.169046 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:55.170458 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:55.171498 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:55.172102 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:55.172189 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:55.174723 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:55.174727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:55.174727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:55.174738 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:55.176494 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:55.176881 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:55.176883 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:55.176884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:55.176979 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:55.177157 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:55.177158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:55.177158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:55.177427 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:55.177555 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:55.177669 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 14:59:55.177761 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1136,"thread_start_us":72,"threads_started":1}
I20260926 14:59:55.178871 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:59:55.190514 28904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:55.180138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53831 (local address 127.7.37.254:40687)
0926 14:59:55.180172 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:55.180174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:55.183917 (+ 3743us) server_negotiation.cc:407] Connection header received
0926 14:59:55.183975 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:55.183976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:55.183984 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:55.184032 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:55.184571 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:55.185530 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:55.186479 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:55.186780 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:55.186825 (+ 45us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:55.188079 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:55.188080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:55.188081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:55.188086 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:55.189462 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:55.189855 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:55.189856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:55.189856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:55.189903 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:55.190089 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:55.190089 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:55.190090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:55.190187 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:55.190262 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:55.190387 (+ 125us) server_negotiation.cc:299] Negotiation successful
0926 14:59:55.190438 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":17}
I20260926 14:59:55.203428 28893 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:55.204102 28893 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:55.210680 28893 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28a32057e6b5455d8a1fa19d270b8bdf: Generated new TSK 0
I20260926 14:59:55.210843 28893 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:55.217640 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 28813
W20260926 14:59:55.221160 28798 connection.cc:535] client connection to 127.7.37.254:40687 recv error: Network error: failed to read from TLS socket (remote: 127.7.37.254:40687): Connection reset by peer (error 104)
2026-09-26T14:59:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1222 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:55 dist-test-slave-f9xb krb5kdc[28912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:55 dist-test-slave-f9xb krb5kdc[28912](info): set up 2 sockets
Sep 26 14:59:55 dist-test-slave-f9xb krb5kdc[28912](info): commencing operation
krb5kdc: starting...
W20260926 14:59:56.159473 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:56 dist-test-slave-f9xb krb5kdc[28912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:59:56.260609 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:37229
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:34969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:37229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:56.334623 28925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:56.334826 28925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:56.334856 28925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:56.335978 28925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:56.336014 28925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:56.336032 28925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:56.336047 28925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:56.336063 28925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:56.337472 28925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:34969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:37229
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:37229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:56.337682 28925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:56.337865 28925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:56.342077 28933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:56.342113 28931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:56.342159 28925 server_base.cc:1034] running on GCE node
W20260926 14:59:56.342082 28930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:56.342495 28925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:56.342760 28925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:56.343888 28925 hybrid_clock.cc:648] HybridClock initialized: now 1790434796343878 us; error 27 us; skew 500 ppm
Sep 26 14:59:56 dist-test-slave-f9xb krb5kdc[28912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434796, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:56.350795 28925 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:59:56.351178 28925 webserver.cc:460] Webserver started at http://127.7.37.254:42449/ using document root <none> and password file <none>
I20260926 14:59:56.351365 28925 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:56.351409 28925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:56.351534 28925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:56.352521 28925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "f8aca82f72e04b64bbc1b61b6bc4c410"
format_stamp: "Formatted at 2026-09-26 14:59:56 on dist-test-slave-f9xb"
server_key: "eb526276b1c36f9f10707138cf48e99d"
server_key_iv: "4d878438ad8eb68f74279485359f1fa1"
server_key_version: "encryptionkey@0"
I20260926 14:59:56.352854 28925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "f8aca82f72e04b64bbc1b61b6bc4c410"
format_stamp: "Formatted at 2026-09-26 14:59:56 on dist-test-slave-f9xb"
server_key: "eb526276b1c36f9f10707138cf48e99d"
server_key_iv: "4d878438ad8eb68f74279485359f1fa1"
server_key_version: "encryptionkey@0"
I20260926 14:59:56.354087 28925 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:56.354692 28940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:56.354846 28925 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 14:59:56.354923 28925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "f8aca82f72e04b64bbc1b61b6bc4c410"
format_stamp: "Formatted at 2026-09-26 14:59:56 on dist-test-slave-f9xb"
server_key: "eb526276b1c36f9f10707138cf48e99d"
server_key_iv: "4d878438ad8eb68f74279485359f1fa1"
server_key_version: "encryptionkey@0"
I20260926 14:59:56.354971 28925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:56.375507 28925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:56.377070 28925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:56.377163 28925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:56.380448 28925 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:37229
I20260926 14:59:56.380482 28992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:37229 every 8 connection(s)
I20260926 14:59:56.380741 28925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:56.381453 28993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:56.383445 28993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410: Bootstrap starting.
I20260926 14:59:56.383940 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 28925
I20260926 14:59:56.383997 28993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:56.384035 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:56.384209 7319 external_mini_cluster.cc:1428] Setting key c178485c9be945b53a5a5b12e562c3b7
I20260926 14:59:56.384255 28993 log.cc:824] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:56.384945 28993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410: No bootstrap required, opened a new log
I20260926 14:59:56.386864 28993 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8aca82f72e04b64bbc1b61b6bc4c410" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37229 } }
I20260926 14:59:56.387015 28993 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:56.387044 28993 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8aca82f72e04b64bbc1b61b6bc4c410, State: Initialized, Role: FOLLOWER
I20260926 14:59:56.387151 28993 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8aca82f72e04b64bbc1b61b6bc4c410" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37229 } }
I20260926 14:59:56.387212 28993 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:56.387253 28993 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:56.387295 28993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:56.387813 28993 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8aca82f72e04b64bbc1b61b6bc4c410" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37229 } }
I20260926 14:59:56.387897 28993 leader_election.cc:302] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8aca82f72e04b64bbc1b61b6bc4c410; no voters:
I20260926 14:59:56.388075 28993 leader_election.cc:288] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:56.388105 28998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:56.388206 28998 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410 [term 1 LEADER]: Becoming Leader. State: Replica: f8aca82f72e04b64bbc1b61b6bc4c410, State: Running, Role: LEADER
I20260926 14:59:56.388321 28993 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:56.388363 28998 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8aca82f72e04b64bbc1b61b6bc4c410" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37229 } }
I20260926 14:59:56.388789 28999 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8aca82f72e04b64bbc1b61b6bc4c410" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8aca82f72e04b64bbc1b61b6bc4c410" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37229 } } }
I20260926 14:59:56.388865 28999 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:56.389050 29000 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8aca82f72e04b64bbc1b61b6bc4c410. Latest consensus state: current_term: 1 leader_uuid: "f8aca82f72e04b64bbc1b61b6bc4c410" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8aca82f72e04b64bbc1b61b6bc4c410" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37229 } } }
I20260926 14:59:56.389145 29000 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:56.389297 29007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:56.389428 28997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:56.384579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47589 (local address 127.7.37.254:37229)
0926 14:59:56.384740 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:56.384746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:56.384769 (+ 23us) server_negotiation.cc:407] Connection header received
0926 14:59:56.385200 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:56.385207 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:56.385249 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:56.385350 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:56.385575 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:56.386412 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:56.387458 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:56.387883 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:56.387978 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:56.388618 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:56.388621 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:56.388622 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:56.388796 (+ 174us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:56.388832 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:56.388846 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:56.388921 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:56.389194 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 14:59:56.389219 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260926 14:59:56.389715 29007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:56.390314 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:59:56.394884 28997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:56.391723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54467 (local address 127.7.37.254:37229)
0926 14:59:56.391767 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:56.391768 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:56.391891 (+ 123us) server_negotiation.cc:407] Connection header received
0926 14:59:56.391947 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:56.391948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:56.391956 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:56.392008 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:56.392139 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:56.392771 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:56.393914 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:56.394112 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:56.394164 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:56.394688 (+ 524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:56.394689 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:56.394690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:56.394693 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:56.394703 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:56.394707 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:56.394747 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:56.394834 (+ 87us) server_negotiation.cc:299] Negotiation successful
0926 14:59:56.394842 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260926 14:59:56.395803 29007 catalog_manager.cc:1347] Generated new cluster ID: 0a88246144d04e02bab95805c9540259
I20260926 14:59:56.395854 29007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:59:56.405216 29007 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:56.405843 29007 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:56.416814 29007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8aca82f72e04b64bbc1b61b6bc4c410: Generated new TSK 0
I20260926 14:59:56.416988 29007 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:56.423990 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 28925
W20260926 14:59:56.427320 28910 connection.cc:535] client connection to 127.7.37.254:37229 recv error: Network error: failed to read from TLS socket (remote: 127.7.37.254:37229): Connection reset by peer (error 104)
2026-09-26T14:59:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1206 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:56 dist-test-slave-f9xb krb5kdc[29024](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:56 dist-test-slave-f9xb krb5kdc[29024](info): set up 2 sockets
Sep 26 14:59:56 dist-test-slave-f9xb krb5kdc[29024](info): commencing operation
krb5kdc: starting...
W20260926 14:59:57.355346 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.915s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:57 dist-test-slave-f9xb krb5kdc[29024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:59:57.456578 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36291
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:36093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:36291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:57.530830 29037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:57.530997 29037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:57.531015 29037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:57.532096 29037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:57.532140 29037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:57.532153 29037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:57.532176 29037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:57.532192 29037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:57.533546 29037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:36093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:36291
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:57.533735 29037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:57.533915 29037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:57.538166 29042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:57.538234 29043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:57.538213 29037 server_base.cc:1034] running on GCE node
W20260926 14:59:57.538174 29045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:57.538507 29037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:57.538759 29037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:57.539885 29037 hybrid_clock.cc:648] HybridClock initialized: now 1790434797539874 us; error 23 us; skew 500 ppm
Sep 26 14:59:57 dist-test-slave-f9xb krb5kdc[29024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434797, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:57.541937 29037 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:59:57.542342 29037 webserver.cc:460] Webserver started at http://127.7.37.254:35057/ using document root <none> and password file <none>
I20260926 14:59:57.542521 29037 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:57.542564 29037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:57.542678 29037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:57.543570 29037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "1e2007b7b57749909695846580609e8c"
format_stamp: "Formatted at 2026-09-26 14:59:57 on dist-test-slave-f9xb"
server_key: "8cd8d3a9999402a3e71158790feb2bed"
server_key_iv: "26f1ab276ce752498740528a4c604289"
server_key_version: "encryptionkey@0"
I20260926 14:59:57.543888 29037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "1e2007b7b57749909695846580609e8c"
format_stamp: "Formatted at 2026-09-26 14:59:57 on dist-test-slave-f9xb"
server_key: "8cd8d3a9999402a3e71158790feb2bed"
server_key_iv: "26f1ab276ce752498740528a4c604289"
server_key_version: "encryptionkey@0"
I20260926 14:59:57.545013 29037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 14:59:57.545652 29052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:57.545799 29037 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:59:57.545871 29037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "1e2007b7b57749909695846580609e8c"
format_stamp: "Formatted at 2026-09-26 14:59:57 on dist-test-slave-f9xb"
server_key: "8cd8d3a9999402a3e71158790feb2bed"
server_key_iv: "26f1ab276ce752498740528a4c604289"
server_key_version: "encryptionkey@0"
I20260926 14:59:57.545931 29037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:57.548904 29037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:57.550587 29037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:57.550702 29037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:57.554067 29037 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:36291
I20260926 14:59:57.554093 29104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:36291 every 8 connection(s)
I20260926 14:59:57.554348 29037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:57.555104 29105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:57.557086 29105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c: Bootstrap starting.
I20260926 14:59:57.557574 29105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:57.557801 29105 log.cc:824] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:57.558369 29105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c: No bootstrap required, opened a new log
I20260926 14:59:57.559800 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 29037
I20260926 14:59:57.559887 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:57.559819 29105 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2007b7b57749909695846580609e8c" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36291 } }
I20260926 14:59:57.559962 29105 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:57.560001 29105 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e2007b7b57749909695846580609e8c, State: Initialized, Role: FOLLOWER
I20260926 14:59:57.560052 7319 external_mini_cluster.cc:1428] Setting key a6f2f983b3be2889cd3b725325c101c7
I20260926 14:59:57.560122 29105 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2007b7b57749909695846580609e8c" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36291 } }
I20260926 14:59:57.560199 29105 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:57.560228 29105 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:57.560271 29105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:57.561157 29105 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2007b7b57749909695846580609e8c" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36291 } }
I20260926 14:59:57.561244 29105 leader_election.cc:302] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e2007b7b57749909695846580609e8c; no voters:
I20260926 14:59:57.561373 29105 leader_election.cc:288] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:57.561404 29110 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:57.561563 29110 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c [term 1 LEADER]: Becoming Leader. State: Replica: 1e2007b7b57749909695846580609e8c, State: Running, Role: LEADER
I20260926 14:59:57.561650 29105 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:57.561710 29110 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2007b7b57749909695846580609e8c" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36291 } }
I20260926 14:59:57.561959 29109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:57.560439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37387 (local address 127.7.37.254:36291)
0926 14:59:57.560590 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:57.560598 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:57.560630 (+ 32us) server_negotiation.cc:407] Connection header received
0926 14:59:57.560761 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:57.560768 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:57.560798 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:57.560934 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:57.561453 (+ 519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:57.561457 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:57.561458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:57.561462 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:57.561504 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:57.561519 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:57.561582 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:57.561733 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 14:59:57.561805 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
I20260926 14:59:57.562060 29110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e2007b7b57749909695846580609e8c. Latest consensus state: current_term: 1 leader_uuid: "1e2007b7b57749909695846580609e8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2007b7b57749909695846580609e8c" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36291 } } }
I20260926 14:59:57.562127 29111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e2007b7b57749909695846580609e8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2007b7b57749909695846580609e8c" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36291 } } }
I20260926 14:59:57.562222 29111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:57.562131 29110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:57.562420 29119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:57.562963 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:59:57.564128 29119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:57.565166 29109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:57.564165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59445 (local address 127.7.37.254:36291)
0926 14:59:57.564193 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:57.564195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:57.564300 (+ 105us) server_negotiation.cc:407] Connection header received
0926 14:59:57.564415 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:57.564415 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:57.564420 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:57.564460 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:57.564983 (+ 523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:57.564984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:57.564984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:57.564988 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:57.564999 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:57.565003 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:57.565044 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:57.565089 (+ 45us) server_negotiation.cc:299] Negotiation successful
0926 14:59:57.565101 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260926 14:59:57.565500 29119 catalog_manager.cc:1347] Generated new cluster ID: a1bc68ef5a7949049cc16f270e43b8a4
I20260926 14:59:57.565584 29119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:59:57.584987 29119 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:57.585111 29119 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:57.591451 29119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e2007b7b57749909695846580609e8c: Generated new TSK 0
I20260926 14:59:57.591583 29119 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:57.597819 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 29037
2026-09-26T14:59:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1173 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:57 dist-test-slave-f9xb krb5kdc[29136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:57 dist-test-slave-f9xb krb5kdc[29136](info): set up 2 sockets
Sep 26 14:59:57 dist-test-slave-f9xb krb5kdc[29136](info): commencing operation
krb5kdc: starting...
W20260926 14:59:58.531461 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.917s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:58 dist-test-slave-f9xb krb5kdc[29136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:59:58.632081 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:43753
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:36483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:43753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:58.707536 29149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:58.707803 29149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:58.707835 29149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:58.708943 29149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:58.708981 29149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:58.708993 29149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:58.709004 29149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:58.709014 29149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:58.710373 29149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:36483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:43753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:43753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:58.710539 29149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:58.710682 29149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:58.715121 29155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:58.715124 29157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:58.715098 29154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:58.715338 29149 server_base.cc:1034] running on GCE node
I20260926 14:59:58.715665 29149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:58.715953 29149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:58.717093 29149 hybrid_clock.cc:648] HybridClock initialized: now 1790434798717082 us; error 30 us; skew 500 ppm
Sep 26 14:59:58 dist-test-slave-f9xb krb5kdc[29136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434798, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:58.719367 29149 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:59:58.719825 29149 webserver.cc:460] Webserver started at http://127.7.37.254:44895/ using document root <none> and password file <none>
I20260926 14:59:58.720017 29149 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:58.720079 29149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:58.720207 29149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:58.721191 29149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "884e5db35d9442d78bbd29f71e04070d"
format_stamp: "Formatted at 2026-09-26 14:59:58 on dist-test-slave-f9xb"
server_key: "130686daa668652a699ab04b7771c3f8"
server_key_iv: "2bafbcc3d6b36e896df1f76f2683bd80"
server_key_version: "encryptionkey@0"
I20260926 14:59:58.721519 29149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "884e5db35d9442d78bbd29f71e04070d"
format_stamp: "Formatted at 2026-09-26 14:59:58 on dist-test-slave-f9xb"
server_key: "130686daa668652a699ab04b7771c3f8"
server_key_iv: "2bafbcc3d6b36e896df1f76f2683bd80"
server_key_version: "encryptionkey@0"
I20260926 14:59:58.722733 29149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 14:59:58.723372 29164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:58.723527 29149 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 14:59:58.723621 29149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "884e5db35d9442d78bbd29f71e04070d"
format_stamp: "Formatted at 2026-09-26 14:59:58 on dist-test-slave-f9xb"
server_key: "130686daa668652a699ab04b7771c3f8"
server_key_iv: "2bafbcc3d6b36e896df1f76f2683bd80"
server_key_version: "encryptionkey@0"
I20260926 14:59:58.723692 29149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:58.746276 29149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:58.747990 29149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:58.748121 29149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:58.751634 29149 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:43753
I20260926 14:59:58.751677 29216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:43753 every 8 connection(s)
I20260926 14:59:58.751966 29149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:58.752708 29217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:58.754755 29217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d: Bootstrap starting.
I20260926 14:59:58.755250 29217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:58.755481 29217 log.cc:824] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:58.755664 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 29149
I20260926 14:59:58.755784 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:58.755934 7319 external_mini_cluster.cc:1428] Setting key 392cacf08c424f0043b09a615d5be9d2
I20260926 14:59:58.756155 29217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d: No bootstrap required, opened a new log
I20260926 14:59:58.758015 29217 raft_consensus.cc:348] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "884e5db35d9442d78bbd29f71e04070d" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43753 } }
I20260926 14:59:58.758191 29217 raft_consensus.cc:374] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:58.758226 29217 raft_consensus.cc:729] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884e5db35d9442d78bbd29f71e04070d, State: Initialized, Role: FOLLOWER
I20260926 14:59:58.758348 29217 consensus_queue.cc:260] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "884e5db35d9442d78bbd29f71e04070d" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43753 } }
I20260926 14:59:58.758416 29217 raft_consensus.cc:388] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:58.758450 29217 raft_consensus.cc:482] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:58.758490 29217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:58.759088 29217 raft_consensus.cc:504] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "884e5db35d9442d78bbd29f71e04070d" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43753 } }
I20260926 14:59:58.759183 29217 leader_election.cc:302] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 884e5db35d9442d78bbd29f71e04070d; no voters:
I20260926 14:59:58.759311 29217 leader_election.cc:288] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:58.759351 29222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:58.759474 29222 raft_consensus.cc:686] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d [term 1 LEADER]: Becoming Leader. State: Replica: 884e5db35d9442d78bbd29f71e04070d, State: Running, Role: LEADER
I20260926 14:59:58.759518 29217 sys_catalog.cc:564] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:58.759552 29222 consensus_queue.cc:237] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "884e5db35d9442d78bbd29f71e04070d" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43753 } }
I20260926 14:59:58.759922 29223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "884e5db35d9442d78bbd29f71e04070d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "884e5db35d9442d78bbd29f71e04070d" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43753 } } }
I20260926 14:59:58.760012 29223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:58.759919 29224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 884e5db35d9442d78bbd29f71e04070d. Latest consensus state: current_term: 1 leader_uuid: "884e5db35d9442d78bbd29f71e04070d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "884e5db35d9442d78bbd29f71e04070d" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 43753 } } }
I20260926 14:59:58.760082 29224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:58.760675 29229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:58.761161 29229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:58.762528 29229 catalog_manager.cc:1347] Generated new cluster ID: 83f7230b8a7b4a1f9824812d55d63e4d
I20260926 14:59:58.762575 29229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 14:59:58 dist-test-slave-f9xb krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 14:59:58.767088 29221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:58.756347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34717 (local address 127.7.37.254:43753)
0926 14:59:58.756532 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:58.756541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:58.756579 (+ 38us) server_negotiation.cc:407] Connection header received
0926 14:59:58.757285 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:58.757294 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:58.757351 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:58.757487 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:58.757992 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:58.759524 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:58.760668 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:58.761298 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:58.761395 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:58.763972 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:58.763976 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:58.763976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:58.763987 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:58.765672 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:58.766037 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:58.766039 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:58.766039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:58.766126 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:58.766307 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:58.766308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:58.766308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:58.766555 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:58.766701 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:58.766803 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 14:59:58.766915 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":90,"threads_started":1}
I20260926 14:59:58.767925 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:59:58.775578 29221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:58.769315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34675 (local address 127.7.37.254:43753)
0926 14:59:58.769347 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:58.769349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:58.769356 (+ 7us) server_negotiation.cc:407] Connection header received
0926 14:59:58.769373 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:58.769374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:58.769382 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:58.769439 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:58.769969 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:58.770691 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:58.771593 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:58.771951 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:58.772016 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:58.773135 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:58.773136 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:58.773137 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 14:59:58.773143 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:58.774534 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:58.774856 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:58.774856 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:58.774857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:58.774910 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 14:59:58.775060 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 14:59:58.775060 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 14:59:58.775060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 14:59:58.775170 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:58.775254 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:58.775339 (+ 85us) server_negotiation.cc:299] Negotiation successful
0926 14:59:58.775404 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 14:59:58.777691 29229 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:58.778162 29229 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:58.785818 29229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 884e5db35d9442d78bbd29f71e04070d: Generated new TSK 0
I20260926 14:59:58.785967 29229 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:58.793628 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 29149
2026-09-26T14:59:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1195 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 14:59:58 dist-test-slave-f9xb krb5kdc[29248](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 14:59:58 dist-test-slave-f9xb krb5kdc[29248](info): set up 2 sockets
Sep 26 14:59:58 dist-test-slave-f9xb krb5kdc[29248](info): commencing operation
krb5kdc: starting...
W20260926 14:59:59.731511 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.922s user 0.004s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 14:59:59 dist-test-slave-f9xb krb5kdc[29248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T14:59:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T14:59:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 14:59:59.834404 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:37359
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:38837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:37359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 14:59:59.909061 29261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 14:59:59.909232 29261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 14:59:59.909261 29261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 14:59:59.910353 29261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 14:59:59.910393 29261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 14:59:59.910413 29261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 14:59:59.910429 29261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 14:59:59.910444 29261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 14:59:59.911878 29261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:38837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:37359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:37359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 14:59:59.912087 29261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 14:59:59.912272 29261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 14:59:59.916528 29266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:59.916520 29269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 14:59:59.916528 29267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 14:59:59.916781 29261 server_base.cc:1034] running on GCE node
I20260926 14:59:59.916953 29261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 14:59:59.917189 29261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 14:59:59.918319 29261 hybrid_clock.cc:648] HybridClock initialized: now 1790434799918306 us; error 27 us; skew 500 ppm
Sep 26 14:59:59 dist-test-slave-f9xb krb5kdc[29248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434799, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 14:59:59.920445 29261 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 14:59:59.920822 29261 webserver.cc:460] Webserver started at http://127.7.37.254:34011/ using document root <none> and password file <none>
I20260926 14:59:59.921020 29261 fs_manager.cc:359] Metadata directory not provided
I20260926 14:59:59.921065 29261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 14:59:59.921192 29261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 14:59:59.922086 29261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "dc2e70d35e0347c399caeb7e7cdd4bb3"
format_stamp: "Formatted at 2026-09-26 14:59:59 on dist-test-slave-f9xb"
server_key: "b996acc3c0f8dffa855b46c7f03931f4"
server_key_iv: "e0b54d0717d11b7fe7c6c3aad3c1b17d"
server_key_version: "encryptionkey@0"
I20260926 14:59:59.922386 29261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "dc2e70d35e0347c399caeb7e7cdd4bb3"
format_stamp: "Formatted at 2026-09-26 14:59:59 on dist-test-slave-f9xb"
server_key: "b996acc3c0f8dffa855b46c7f03931f4"
server_key_iv: "e0b54d0717d11b7fe7c6c3aad3c1b17d"
server_key_version: "encryptionkey@0"
I20260926 14:59:59.923532 29261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 14:59:59.924213 29276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:59.924350 29261 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 14:59:59.924427 29261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "dc2e70d35e0347c399caeb7e7cdd4bb3"
format_stamp: "Formatted at 2026-09-26 14:59:59 on dist-test-slave-f9xb"
server_key: "b996acc3c0f8dffa855b46c7f03931f4"
server_key_iv: "e0b54d0717d11b7fe7c6c3aad3c1b17d"
server_key_version: "encryptionkey@0"
I20260926 14:59:59.924489 29261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 14:59:59.934331 29261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 14:59:59.935858 29261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 14:59:59.935976 29261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 14:59:59.939342 29261 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:37359
I20260926 14:59:59.939371 29328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:37359 every 8 connection(s)
I20260926 14:59:59.939636 29261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 14:59:59.940397 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 14:59:59.942253 29329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3: Bootstrap starting.
I20260926 14:59:59.942731 29329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3: Neither blocks nor log segments found. Creating new log.
I20260926 14:59:59.942966 29329 log.cc:824] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3: Log is configured to *not* fsync() on all Append() calls
I20260926 14:59:59.943531 29329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3: No bootstrap required, opened a new log
I20260926 14:59:59.944968 29329 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc2e70d35e0347c399caeb7e7cdd4bb3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37359 } }
I20260926 14:59:59.945084 29329 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 14:59:59.945104 29329 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc2e70d35e0347c399caeb7e7cdd4bb3, State: Initialized, Role: FOLLOWER
I20260926 14:59:59.945227 29329 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc2e70d35e0347c399caeb7e7cdd4bb3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37359 } }
I20260926 14:59:59.945297 29329 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 14:59:59.945319 29329 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 14:59:59.945364 29329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 14:59:59.946076 29329 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc2e70d35e0347c399caeb7e7cdd4bb3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37359 } }
I20260926 14:59:59.946162 29329 leader_election.cc:302] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc2e70d35e0347c399caeb7e7cdd4bb3; no voters:
I20260926 14:59:59.946300 29329 leader_election.cc:288] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 14:59:59.946337 29332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 14:59:59.946489 29332 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3 [term 1 LEADER]: Becoming Leader. State: Replica: dc2e70d35e0347c399caeb7e7cdd4bb3, State: Running, Role: LEADER
I20260926 14:59:59.946496 29329 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 14:59:59.946641 29332 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc2e70d35e0347c399caeb7e7cdd4bb3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37359 } }
I20260926 14:59:59.946971 29333 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc2e70d35e0347c399caeb7e7cdd4bb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc2e70d35e0347c399caeb7e7cdd4bb3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37359 } } }
I20260926 14:59:59.946986 29334 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc2e70d35e0347c399caeb7e7cdd4bb3. Latest consensus state: current_term: 1 leader_uuid: "dc2e70d35e0347c399caeb7e7cdd4bb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc2e70d35e0347c399caeb7e7cdd4bb3" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37359 } } }
I20260926 14:59:59.947062 29333 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:59.947072 29334 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3 [sys.catalog]: This master's current role is: LEADER
I20260926 14:59:59.947499 29341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 14:59:59.947643 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 29261
I20260926 14:59:59.947774 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 14:59:59.947944 7319 external_mini_cluster.cc:1428] Setting key 93bc86e9ead2f5d0af716cedda131bde
I20260926 14:59:59.947981 29341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 14:59:59.949394 29341 catalog_manager.cc:1347] Generated new cluster ID: 842bd11c5bb045c58640afa9cd754740
I20260926 14:59:59.949443 29341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 14:59:59.952421 29351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:59.948447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59837 (local address 127.7.37.254:37359)
0926 14:59:59.948613 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:59.948621 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:59.948649 (+ 28us) server_negotiation.cc:407] Connection header received
0926 14:59:59.948747 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:59.948753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:59.948801 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:59.949164 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:59.949234 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:59.950152 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:59.950866 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:59.951257 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:59.951350 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:59.951894 (+ 544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:59.951897 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:59.951898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:59.952042 (+ 144us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:59.952068 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:59.952078 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:59.952138 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:59.952209 (+ 71us) server_negotiation.cc:299] Negotiation successful
0926 14:59:59.952265 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260926 14:59:59.953156 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 14:59:59.957037 29351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 14:59:59.954320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47101 (local address 127.7.37.254:37359)
0926 14:59:59.954354 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 14:59:59.954355 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 14:59:59.954447 (+ 92us) server_negotiation.cc:407] Connection header received
0926 14:59:59.954472 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 14:59:59.954473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 14:59:59.954481 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 14:59:59.954533 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 14:59:59.954615 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:59.955295 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:59.956022 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 14:59:59.956307 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 14:59:59.956357 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 14:59:59.956858 (+ 501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 14:59:59.956859 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 14:59:59.956860 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 14:59:59.956863 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 14:59:59.956872 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 14:59:59.956875 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 14:59:59.956920 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 14:59:59.956956 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 14:59:59.956973 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 14:59:59.966966 29341 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 14:59:59.967404 29341 catalog_manager.cc:1504] Loading token signing keys...
I20260926 14:59:59.975725 29341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc2e70d35e0347c399caeb7e7cdd4bb3: Generated new TSK 0
I20260926 14:59:59.975867 29341 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 14:59:59.978662 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 29261
2026-09-26T14:59:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1184 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:00:00 dist-test-slave-f9xb krb5kdc[29360](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:00:00 dist-test-slave-f9xb krb5kdc[29360](info): set up 2 sockets
Sep 26 15:00:00 dist-test-slave-f9xb krb5kdc[29360](info): commencing operation
krb5kdc: starting...
W20260926 15:00:00.907439 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.913s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 15:00:00 dist-test-slave-f9xb krb5kdc[29360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:00:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:00:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 15:00:01.008721 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:37867
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:45961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:37867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:00:01.090037 29373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:00:01.090229 29373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:00:01.090258 29373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:00:01.091357 29373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:00:01.091394 29373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:00:01.091413 29373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:00:01.091428 29373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:00:01.091444 29373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:00:01.092860 29373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:45961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:37867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:37867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 15:00:01.093091 29373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:00:01.093276 29373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:00:01.097561 29378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:00:01.097751 29373 server_base.cc:1034] running on GCE node
W20260926 15:00:01.097579 29379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:00:01.097589 29381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:00:01.098059 29373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:00:01.098313 29373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:00:01.099445 29373 hybrid_clock.cc:648] HybridClock initialized: now 1790434801099428 us; error 31 us; skew 500 ppm
Sep 26 15:00:01 dist-test-slave-f9xb krb5kdc[29360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434801, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:00:01.101706 29373 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 15:00:01.102064 29373 webserver.cc:460] Webserver started at http://127.7.37.254:41117/ using document root <none> and password file <none>
I20260926 15:00:01.102238 29373 fs_manager.cc:359] Metadata directory not provided
I20260926 15:00:01.102283 29373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:00:01.102401 29373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:00:01.103305 29373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "f165d1c8143e48bab5e9bccd7d70eb68"
format_stamp: "Formatted at 2026-09-26 15:00:01 on dist-test-slave-f9xb"
server_key: "49b1d9500883b21c15a2aa50af437a85"
server_key_iv: "843b885a369504a39afc0830621c466d"
server_key_version: "encryptionkey@0"
I20260926 15:00:01.103601 29373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "f165d1c8143e48bab5e9bccd7d70eb68"
format_stamp: "Formatted at 2026-09-26 15:00:01 on dist-test-slave-f9xb"
server_key: "49b1d9500883b21c15a2aa50af437a85"
server_key_iv: "843b885a369504a39afc0830621c466d"
server_key_version: "encryptionkey@0"
I20260926 15:00:01.104830 29373 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 15:00:01.105506 29388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:00:01.105660 29373 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 15:00:01.105746 29373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "f165d1c8143e48bab5e9bccd7d70eb68"
format_stamp: "Formatted at 2026-09-26 15:00:01 on dist-test-slave-f9xb"
server_key: "49b1d9500883b21c15a2aa50af437a85"
server_key_iv: "843b885a369504a39afc0830621c466d"
server_key_version: "encryptionkey@0"
I20260926 15:00:01.105809 29373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:00:01.108861 29373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:00:01.110920 29373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:00:01.111042 29373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:00:01.114482 29373 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:37867
I20260926 15:00:01.114512 29440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:37867 every 8 connection(s)
I20260926 15:00:01.114776 29373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 15:00:01.115602 29441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:00:01.117648 29441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68: Bootstrap starting.
I20260926 15:00:01.118150 29441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68: Neither blocks nor log segments found. Creating new log.
I20260926 15:00:01.118391 29441 log.cc:824] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68: Log is configured to *not* fsync() on all Append() calls
I20260926 15:00:01.118959 29441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68: No bootstrap required, opened a new log
I20260926 15:00:01.120497 29441 raft_consensus.cc:348] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f165d1c8143e48bab5e9bccd7d70eb68" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37867 } }
I20260926 15:00:01.120621 29441 raft_consensus.cc:374] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:00:01.120641 29441 raft_consensus.cc:729] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f165d1c8143e48bab5e9bccd7d70eb68, State: Initialized, Role: FOLLOWER
I20260926 15:00:01.120769 29441 consensus_queue.cc:260] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f165d1c8143e48bab5e9bccd7d70eb68" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37867 } }
I20260926 15:00:01.120836 29441 raft_consensus.cc:388] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:00:01.120867 29441 raft_consensus.cc:482] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:00:01.120911 29441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:00:01.121644 29441 raft_consensus.cc:504] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f165d1c8143e48bab5e9bccd7d70eb68" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37867 } }
I20260926 15:00:01.121726 29441 leader_election.cc:302] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f165d1c8143e48bab5e9bccd7d70eb68; no voters:
I20260926 15:00:01.121867 29441 leader_election.cc:288] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:00:01.121903 29444 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:00:01.122033 29444 raft_consensus.cc:686] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68 [term 1 LEADER]: Becoming Leader. State: Replica: f165d1c8143e48bab5e9bccd7d70eb68, State: Running, Role: LEADER
I20260926 15:00:01.122087 29441 sys_catalog.cc:564] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:00:01.122180 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 29373
I20260926 15:00:01.122211 29444 consensus_queue.cc:237] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f165d1c8143e48bab5e9bccd7d70eb68" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37867 } }
I20260926 15:00:01.122280 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 15:00:01.122452 7319 external_mini_cluster.cc:1428] Setting key 639bf37a22a998363f88807a856950af
I20260926 15:00:01.122583 29445 sys_catalog.cc:455] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f165d1c8143e48bab5e9bccd7d70eb68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f165d1c8143e48bab5e9bccd7d70eb68" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37867 } } }
I20260926 15:00:01.122646 29445 sys_catalog.cc:458] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68 [sys.catalog]: This master's current role is: LEADER
I20260926 15:00:01.122784 29446 sys_catalog.cc:455] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f165d1c8143e48bab5e9bccd7d70eb68. Latest consensus state: current_term: 1 leader_uuid: "f165d1c8143e48bab5e9bccd7d70eb68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f165d1c8143e48bab5e9bccd7d70eb68" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 37867 } } }
I20260926 15:00:01.122833 29446 sys_catalog.cc:458] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68 [sys.catalog]: This master's current role is: LEADER
I20260926 15:00:01.123119 29453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:00:01.123592 29453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:00:01.125089 29455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:01.122847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42115 (local address 127.7.37.254:37867)
0926 15:00:01.123019 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:01.123028 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:01.123569 (+ 541us) server_negotiation.cc:407] Connection header received
0926 15:00:01.123857 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:01.123865 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:01.123897 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:01.124022 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:01.124679 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:01.124683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:01.124683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:00:01.124687 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:01.124725 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:00:01.124739 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:01.124915 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:01.124938 (+ 23us) server_negotiation.cc:299] Negotiation successful
0926 15:00:01.124970 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":60,"threads_started":1}
I20260926 15:00:01.125284 29453 catalog_manager.cc:1347] Generated new cluster ID: 1c8168f5ae0e4a7382facffb95c7aac9
I20260926 15:00:01.125348 29453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:00:01.125891 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:00:01.129102 29455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:01.128093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48779 (local address 127.7.37.254:37867)
0926 15:00:01.128123 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:01.128125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:01.128308 (+ 183us) server_negotiation.cc:407] Connection header received
0926 15:00:01.128360 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:01.128361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:01.128365 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:01.128409 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:01.128909 (+ 500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:01.128910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:01.128910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:00:01.128914 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:01.128926 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:00:01.128929 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:01.128973 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:01.129020 (+ 47us) server_negotiation.cc:299] Negotiation successful
0926 15:00:01.129032 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260926 15:00:01.132143 29453 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:00:01.132278 29453 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:00:01.135735 29453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f165d1c8143e48bab5e9bccd7d70eb68: Generated new TSK 0
I20260926 15:00:01.135895 29453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:00:01.139626 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 29373
2026-09-26T15:00:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1161 ms)
[----------] 18 tests from AuthTokenIssuingTest (21432 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:00:01 dist-test-slave-f9xb krb5kdc[29472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:00:01 dist-test-slave-f9xb krb5kdc[29472](info): set up 2 sockets
Sep 26 15:00:01 dist-test-slave-f9xb krb5kdc[29472](info): commencing operation
krb5kdc: starting...
W20260926 15:00:02.076383 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.920s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 15:00:02 dist-test-slave-f9xb krb5kdc[29472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:00:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:00:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 15:00:02.182713 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36259
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:36259,127.7.37.253:43517,127.7.37.252:44571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:00:02.257521 29485 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 15:00:02.257717 29485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:00:02.257733 29485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:00:02.257751 29485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:00:02.258828 29485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:00:02.258865 29485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:00:02.258877 29485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:00:02.258889 29485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:00:02.258912 29485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:00:02.260318 29485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:36259,127.7.37.253:43517,127.7.37.252:44571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:36259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 15:00:02.260524 29485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:00:02.260712 29485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:00:02.264945 29490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:00:02.264968 29491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:00:02.264942 29493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:00:02.265138 29485 server_base.cc:1034] running on GCE node
I20260926 15:00:02.265337 29485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:00:02.265563 29485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:00:02.266702 29485 hybrid_clock.cc:648] HybridClock initialized: now 1790434802266693 us; error 26 us; skew 500 ppm
Sep 26 15:00:02 dist-test-slave-f9xb krb5kdc[29472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:00:02.272882 29485 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 15:00:02.273281 29485 webserver.cc:460] Webserver started at http://127.7.37.254:41689/ using document root <none> and password file <none>
I20260926 15:00:02.273451 29485 fs_manager.cc:359] Metadata directory not provided
I20260926 15:00:02.273504 29485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:00:02.273617 29485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:00:02.274530 29485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "28286e867f674d08b2c72991cc2d2d93"
format_stamp: "Formatted at 2026-09-26 15:00:02 on dist-test-slave-f9xb"
server_key: "a285436580dcd7ff926b86e08346205a"
server_key_iv: "f98ee1ddda0be323d6cb40e9420bac4d"
server_key_version: "encryptionkey@0"
I20260926 15:00:02.274838 29485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "28286e867f674d08b2c72991cc2d2d93"
format_stamp: "Formatted at 2026-09-26 15:00:02 on dist-test-slave-f9xb"
server_key: "a285436580dcd7ff926b86e08346205a"
server_key_iv: "f98ee1ddda0be323d6cb40e9420bac4d"
server_key_version: "encryptionkey@0"
I20260926 15:00:02.276113 29485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 15:00:02.276805 29500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:00:02.276952 29485 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 15:00:02.277024 29485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "28286e867f674d08b2c72991cc2d2d93"
format_stamp: "Formatted at 2026-09-26 15:00:02 on dist-test-slave-f9xb"
server_key: "a285436580dcd7ff926b86e08346205a"
server_key_iv: "f98ee1ddda0be323d6cb40e9420bac4d"
server_key_version: "encryptionkey@0"
I20260926 15:00:02.277088 29485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:00:02.297698 29485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:00:02.299393 29485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:00:02.299510 29485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:00:02.303158 29485 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:36259
I20260926 15:00:02.303200 29552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:36259 every 8 connection(s)
I20260926 15:00:02.303483 29485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 15:00:02.304474 29553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:00:02.305962 29553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:00:02.306373 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 29485
I20260926 15:00:02.306468 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 15:00:02.306641 7319 external_mini_cluster.cc:1428] Setting key 88af694faaf6fdd5b841accaa96c0a70
Sep 26 15:00:02 dist-test-slave-f9xb krb5kdc[29472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.37.253@KRBTEST.COM; defaulting to no policy
I20260926 15:00:02.315490 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.306618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36259 (local address 127.0.0.1:35800)
0926 15:00:02.306790 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.306826 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 15:00:02.307250 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:00:02.307535 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:00:02.307537 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:00:02.307800 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:00:02.308019 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:02.308024 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.308993 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.308994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:02.310035 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:02.310038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.310513 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.310514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:02.310597 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.311046 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:00:02.311056 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:00:02.312526 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:00:02.314209 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:02.314215 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:02.314217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:02.314400 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:02.314628 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:02.314629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:02.314630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:02.314714 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:02.315019 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:00:02.315022 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:00:02.315069 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 15:00:02.315133 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 15:00:02.315360 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":98,"thread_start_us":85,"threads_started":1}
I20260926 15:00:02.315497 29555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.306630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35800 (local address 127.7.37.254:36259)
0926 15:00:02.306804 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.306813 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.306902 (+ 89us) server_negotiation.cc:407] Connection header received
0926 15:00:02.307394 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.307396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.307417 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.307516 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.308112 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.308901 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.310122 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.310418 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.310493 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.312609 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:02.312616 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:02.312616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:02.312623 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:02.314132 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.314471 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.314474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.314474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.314563 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.314771 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.314772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.314772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.314966 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:02.315033 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.315163 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.315282 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":139,"thread_start_us":79,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.253@KRBTEST.COM" created.
I20260926 15:00:02.316236 29553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:00:02.316560 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.316442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:43517 (local address 127.0.0.1:37102)
0926 15:00:02.316486 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.316535 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 15:00:02.316699 29503 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43517: connect: Connection refused (error 111)
I20260926 15:00:02.317082 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.316981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:43517 (local address 127.0.0.1:37116)
0926 15:00:02.317021 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.317060 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260926 15:00:02.317178 29553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.253:43517: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43517: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.253.keytab.
Entry for principal kudu/127.7.37.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.253.keytab.
I20260926 15:00:02.328224 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.253:43517
--webserver_interface=127.7.37.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.253.keytab
--principal=kudu/127.7.37.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:36259,127.7.37.253:43517,127.7.37.252:44571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:00:02.366390 29553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } attempt: 1
I20260926 15:00:02.366876 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.366758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:43517 (local address 127.0.0.1:37118)
0926 15:00:02.366811 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.366850 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 15:00:02.367233 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.367142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:43517 (local address 127.0.0.1:37134)
0926 15:00:02.367186 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.367210 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260926 15:00:02.367316 29553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.253:43517: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43517: connect: Connection refused (error 111)
W20260926 15:00:02.404105 29560 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 15:00:02.404285 29560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:00:02.404300 29560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:00:02.404318 29560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:00:02.405350 29560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:00:02.405382 29560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:00:02.405395 29560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:00:02.405416 29560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:00:02.405429 29560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:00:02.406791 29560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:36259,127.7.37.253:43517,127.7.37.252:44571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.253.keytab
--principal=kudu/127.7.37.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.253:43517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.37.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 15:00:02.406958 29560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:00:02.407116 29560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:00:02.411394 29566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:00:02.411453 29565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:00:02.411592 29560 server_base.cc:1034] running on GCE node
W20260926 15:00:02.411389 29568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:00:02.411862 29560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:00:02.412130 29560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:00:02.413271 29560 hybrid_clock.cc:648] HybridClock initialized: now 1790434802413259 us; error 28 us; skew 500 ppm
Sep 26 15:00:02 dist-test-slave-f9xb krb5kdc[29472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:00:02.415553 29560 init.cc:377] Logged in from keytab as kudu/127.7.37.253@KRBTEST.COM (short username kudu)
I20260926 15:00:02.415962 29560 webserver.cc:460] Webserver started at http://127.7.37.253:46555/ using document root <none> and password file <none>
I20260926 15:00:02.416157 29560 fs_manager.cc:359] Metadata directory not provided
I20260926 15:00:02.416201 29560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:00:02.416312 29560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:00:02.417245 29560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-1/data/instance:
uuid: "6ef39407bb3c4f5c82d7cc9d31194427"
format_stamp: "Formatted at 2026-09-26 15:00:02 on dist-test-slave-f9xb"
server_key: "4a1f5bd74b68ecdc0e83f88b041aad55"
server_key_iv: "6a4c804215df96b184dc166ec460bd5f"
server_key_version: "encryptionkey@0"
I20260926 15:00:02.417534 29560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-1/wal/instance:
uuid: "6ef39407bb3c4f5c82d7cc9d31194427"
format_stamp: "Formatted at 2026-09-26 15:00:02 on dist-test-slave-f9xb"
server_key: "4a1f5bd74b68ecdc0e83f88b041aad55"
server_key_iv: "6a4c804215df96b184dc166ec460bd5f"
server_key_version: "encryptionkey@0"
I20260926 15:00:02.418702 29560 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 15:00:02.419363 29575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:00:02.419512 29560 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 15:00:02.419601 29560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-1/wal
uuid: "6ef39407bb3c4f5c82d7cc9d31194427"
format_stamp: "Formatted at 2026-09-26 15:00:02 on dist-test-slave-f9xb"
server_key: "4a1f5bd74b68ecdc0e83f88b041aad55"
server_key_iv: "6a4c804215df96b184dc166ec460bd5f"
server_key_version: "encryptionkey@0"
I20260926 15:00:02.419665 29560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:00:02.435477 29553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } attempt: 2
I20260926 15:00:02.435887 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.435732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:43517 (local address 127.0.0.1:37150)
0926 15:00:02.435811 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.435850 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 15:00:02.436264 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.436147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:43517 (local address 127.0.0.1:37152)
0926 15:00:02.436199 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.436233 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 15:00:02.436368 29553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.253:43517: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43517: connect: Connection refused (error 111)
I20260926 15:00:02.437147 29560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:00:02.438771 29560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:00:02.438877 29560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:00:02.442689 29560 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.253:43517
I20260926 15:00:02.442719 29627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.253:43517 every 8 connection(s)
I20260926 15:00:02.443022 29560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-1/data/info.pb
I20260926 15:00:02.443890 29628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:00:02.445269 29628 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:00:02 dist-test-slave-f9xb krb5kdc[29472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.253@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 15:00:02.451930 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 29560
I20260926 15:00:02.452033 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-1/wal/instance
I20260926 15:00:02.452252 7319 external_mini_cluster.cc:1428] Setting key 603571fd6142c6f624a9d2a12e30877f
I20260926 15:00:02.452867 29555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.445827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35816 (local address 127.7.37.254:36259)
0926 15:00:02.445879 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.445881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.446042 (+ 161us) server_negotiation.cc:407] Connection header received
0926 15:00:02.446455 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.446455 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.446465 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.446521 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.447069 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.447531 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.448488 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.448687 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.448737 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.450531 (+ 1794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:02.450532 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:02.450533 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:02.450539 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:02.451579 (+ 1040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.451983 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.451984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.451985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.452039 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.452313 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.452313 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.452314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.452419 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:02.452466 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.452754 (+ 288us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.452810 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 15:00:02.453071 29630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.445799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36259 (local address 127.0.0.1:35816)
0926 15:00:02.445934 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.445970 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 15:00:02.446379 (+ 409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:00:02.446576 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:00:02.446579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:00:02.446798 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:00:02.446974 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:02.446977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.447603 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.447603 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:02.448406 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:02.448408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.448756 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.448756 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:02.448842 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.449250 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:00:02.449257 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:00:02.450455 (+ 1198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:00:02.451645 (+ 1190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:02.451650 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:02.451651 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:02.451920 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:02.452134 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:02.452134 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:02.452135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:02.452259 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:02.452475 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:00:02.452481 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:00:02.452619 (+ 138us) client_negotiation.cc:769] Sending connection context
0926 15:00:02.452735 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 15:00:02.452878 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":67,"threads_started":1}
I20260926 15:00:02.453538 29628 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:00:02 dist-test-slave-f9xb krb5kdc[29472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.253@KRBTEST.COM for kudu/127.7.37.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.37.252@KRBTEST.COM; defaulting to no policy
I20260926 15:00:02.460350 29630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.453749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:43517 (local address 127.0.0.1:37156)
0926 15:00:02.453792 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.453803 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 15:00:02.453843 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:00:02.454013 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:00:02.454014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:00:02.454152 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:00:02.454204 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:02.454206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.455232 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.455233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:02.455717 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:02.455719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.456384 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.456385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:02.456445 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.456806 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:00:02.456810 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:00:02.457854 (+ 1044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:00:02.459568 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:02.459569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:02.459569 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:02.459723 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:02.459925 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:02.459926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:02.459926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:02.459970 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:02.460156 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:00:02.460156 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:00:02.460186 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 15:00:02.460225 (+ 39us) client_negotiation.cc:240] Negotiation successful
0926 15:00:02.460290 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260926 15:00:02.460433 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.453782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37156 (local address 127.7.37.253:43517)
0926 15:00:02.453897 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.453901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.453917 (+ 16us) server_negotiation.cc:407] Connection header received
0926 15:00:02.453937 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.453938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.453948 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.454003 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.454267 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.455149 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.455818 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.456295 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.456368 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.457933 (+ 1565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:02.457941 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:02.457941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:02.457948 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:02.459481 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.459820 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.459823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.459824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.459879 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.460004 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.460004 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.460004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.460110 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:02.460165 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.460251 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.460338 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":91,"thread_start_us":59,"threads_started":1}
I20260926 15:00:02.460879 29628 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:00:02.461155 29630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.461054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:44571 (local address 127.0.0.1:40112)
0926 15:00:02.461094 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.461136 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 15:00:02.461277 29577 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.37.252:44571: connect: Connection refused (error 111)
I20260926 15:00:02.461654 29630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.461578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:44571 (local address 127.0.0.1:40118)
0926 15:00:02.461601 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.461635 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260926 15:00:02.461760 29628 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.252:44571: Network error: Client connection negotiation failed: client connection to 127.7.37.252:44571: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.252.keytab.
Entry for principal kudu/127.7.37.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.252.keytab.
I20260926 15:00:02.474996 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.252:44571
--webserver_interface=127.7.37.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.252.keytab
--principal=kudu/127.7.37.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:36259,127.7.37.253:43517,127.7.37.252:44571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:00:02.510962 29628 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } attempt: 1
I20260926 15:00:02.511456 29630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.511287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:44571 (local address 127.0.0.1:40126)
0926 15:00:02.511376 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.511420 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260926 15:00:02.511852 29630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.511772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:44571 (local address 127.0.0.1:40128)
0926 15:00:02.511801 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.511829 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260926 15:00:02.511948 29628 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.252:44571: Network error: Client connection negotiation failed: client connection to 127.7.37.252:44571: connect: Connection refused (error 111)
I20260926 15:00:02.527495 29553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } attempt: 3
Sep 26 15:00:02 dist-test-slave-f9xb krb5kdc[29472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for kudu/127.7.37.253@KRBTEST.COM
I20260926 15:00:02.533850 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.527823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:43517 (local address 127.0.0.1:37168)
0926 15:00:02.527863 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.527873 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 15:00:02.527926 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:00:02.528048 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:00:02.528049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:00:02.528207 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:00:02.528269 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:02.528270 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.528901 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.528902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:02.529439 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:02.529441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.529812 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.529813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:02.529886 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.530276 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:00:02.530281 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:00:02.531568 (+ 1287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:00:02.532986 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:02.532987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:02.532988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:02.533143 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:02.533303 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:02.533303 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:02.533304 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:02.533357 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:02.533556 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:00:02.533557 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:00:02.533600 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 15:00:02.533661 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 15:00:02.533749 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260926 15:00:02.533849 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.527850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37168 (local address 127.7.37.253:43517)
0926 15:00:02.527914 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.527916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.527926 (+ 10us) server_negotiation.cc:407] Connection header received
0926 15:00:02.527979 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.527980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.527992 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.528038 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.528342 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.528831 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.529517 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.529741 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.529807 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.531657 (+ 1850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:02.531659 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:02.531659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:02.531666 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:02.532917 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.533202 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.533203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.533203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.533252 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.533405 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.533406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.533406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.533507 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:02.533559 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.533687 (+ 128us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.533752 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 15:00:02.534391 29553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:00:02.534695 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.534609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:44571 (local address 127.0.0.1:40140)
0926 15:00:02.534649 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.534676 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260926 15:00:02.534991 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.534911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:44571 (local address 127.0.0.1:40154)
0926 15:00:02.534947 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.534968 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260926 15:00:02.535068 29553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.252:44571: Network error: Client connection negotiation failed: client connection to 127.7.37.252:44571: connect: Connection refused (error 111)
W20260926 15:00:02.552152 29635 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 15:00:02.552340 29635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:00:02.552362 29635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:00:02.552388 29635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:00:02.553470 29635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:00:02.553509 29635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:00:02.553527 29635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:00:02.553542 29635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:00:02.553558 29635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:00:02.554968 29635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:36259,127.7.37.253:43517,127.7.37.252:44571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.252.keytab
--principal=kudu/127.7.37.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.252:44571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.37.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 15:00:02.555186 29635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:00:02.555377 29635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:00:02.559823 29643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:00:02.559814 29640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:00:02.559800 29641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:00:02.559930 29635 server_base.cc:1034] running on GCE node
I20260926 15:00:02.560248 29635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:00:02.560497 29635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:00:02.561636 29635 hybrid_clock.cc:648] HybridClock initialized: now 1790434802561621 us; error 25 us; skew 500 ppm
Sep 26 15:00:02 dist-test-slave-f9xb krb5kdc[29472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:00:02.563920 29635 init.cc:377] Logged in from keytab as kudu/127.7.37.252@KRBTEST.COM (short username kudu)
I20260926 15:00:02.564316 29635 webserver.cc:460] Webserver started at http://127.7.37.252:37297/ using document root <none> and password file <none>
I20260926 15:00:02.564520 29635 fs_manager.cc:359] Metadata directory not provided
I20260926 15:00:02.564566 29635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:00:02.564685 29635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:00:02.565618 29635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-2/data/instance:
uuid: "609a800593b8431d9543adc8026f3ae6"
format_stamp: "Formatted at 2026-09-26 15:00:02 on dist-test-slave-f9xb"
server_key: "417f77f554dfb862b41641c60a98c8b9"
server_key_iv: "4c78ee1ddc9dfb953df6b694bd041648"
server_key_version: "encryptionkey@0"
I20260926 15:00:02.565933 29635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-2/wal/instance:
uuid: "609a800593b8431d9543adc8026f3ae6"
format_stamp: "Formatted at 2026-09-26 15:00:02 on dist-test-slave-f9xb"
server_key: "417f77f554dfb862b41641c60a98c8b9"
server_key_iv: "4c78ee1ddc9dfb953df6b694bd041648"
server_key_version: "encryptionkey@0"
I20260926 15:00:02.566200 29553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } attempt: 1
I20260926 15:00:02.566560 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.566425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:44571 (local address 127.0.0.1:40166)
0926 15:00:02.566487 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.566524 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 15:00:02.566922 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.566833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:44571 (local address 127.0.0.1:40170)
0926 15:00:02.566875 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.566901 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 15:00:02.567001 29553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.252:44571: Network error: Client connection negotiation failed: client connection to 127.7.37.252:44571: connect: Connection refused (error 111)
I20260926 15:00:02.567437 29635 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 15:00:02.568116 29650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:00:02.568287 29635 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 15:00:02.568362 29635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-2/wal
uuid: "609a800593b8431d9543adc8026f3ae6"
format_stamp: "Formatted at 2026-09-26 15:00:02 on dist-test-slave-f9xb"
server_key: "417f77f554dfb862b41641c60a98c8b9"
server_key_iv: "4c78ee1ddc9dfb953df6b694bd041648"
server_key_version: "encryptionkey@0"
I20260926 15:00:02.568427 29635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:00:02.580088 29628 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } attempt: 2
I20260926 15:00:02.580442 29630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.580320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:44571 (local address 127.0.0.1:40184)
0926 15:00:02.580371 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.580407 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 15:00:02.580767 29630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.580680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:44571 (local address 127.0.0.1:40196)
0926 15:00:02.580716 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.580743 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260926 15:00:02.580852 29628 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.252:44571: Network error: Client connection negotiation failed: client connection to 127.7.37.252:44571: connect: Connection refused (error 111)
I20260926 15:00:02.581650 29635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:00:02.583189 29635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:00:02.583302 29635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:00:02.586872 29635 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.252:44571
I20260926 15:00:02.586903 29702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.252:44571 every 8 connection(s)
I20260926 15:00:02.587193 29635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-2/data/info.pb
I20260926 15:00:02.588074 29703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:00:02.588542 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 29635
I20260926 15:00:02.588631 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434555014799-7319-0/minicluster-data/master-2/wal/instance
I20260926 15:00:02.588814 7319 external_mini_cluster.cc:1428] Setting key 6b555ddf7ef592489e3c6bec20b2e293
I20260926 15:00:02.589671 29703 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:00:02 dist-test-slave-f9xb krb5kdc[29472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.252@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
Sep 26 15:00:02 dist-test-slave-f9xb krb5kdc[29472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.252@KRBTEST.COM
I20260926 15:00:02.596917 29555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.590365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35818 (local address 127.7.37.254:36259)
0926 15:00:02.590411 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.590413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.590565 (+ 152us) server_negotiation.cc:407] Connection header received
0926 15:00:02.590632 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.590633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.590643 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.590688 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.591120 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.591555 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.592387 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.592562 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.592615 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.594524 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:02.594525 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:02.594525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:02.594532 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:02.595684 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.596091 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.596092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.596093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.596150 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.596343 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.596344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.596344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.596470 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:02.596535 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.596747 (+ 212us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.596828 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 15:00:02.597026 29707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.590305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:36259 (local address 127.0.0.1:35818)
0926 15:00:02.590488 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.590511 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 15:00:02.590566 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:00:02.590709 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:00:02.590711 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:00:02.590951 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:00:02.591040 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:02.591044 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.591626 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.591627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:02.592317 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:02.592319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.592643 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.592644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:02.592711 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.593224 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:00:02.593231 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:00:02.594450 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:00:02.595733 (+ 1283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:02.595754 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:02.595755 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:02.596013 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:02.596210 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:02.596210 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:02.596210 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:02.596292 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:02.596524 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:00:02.596526 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:00:02.596628 (+ 102us) client_negotiation.cc:769] Sending connection context
0926 15:00:02.596709 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 15:00:02.596834 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":76,"threads_started":1}
I20260926 15:00:02.597534 29706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.589236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40198 (local address 127.7.37.252:44571)
0926 15:00:02.589381 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.589388 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.589410 (+ 22us) server_negotiation.cc:407] Connection header received
0926 15:00:02.589791 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.589809 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.589852 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.589954 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.590635 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.591462 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.592218 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.592516 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.592612 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.594984 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:02.594990 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:02.594990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:02.594996 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:02.596560 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.596847 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.596849 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.596850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.596904 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.597056 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.597056 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.597057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.597185 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:02.597303 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.597382 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.597446 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
I20260926 15:00:02.597671 29703 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:00:02 dist-test-slave-f9xb krb5kdc[29472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.252@KRBTEST.COM for kudu/127.7.37.253@KRBTEST.COM
Sep 26 15:00:02 dist-test-slave-f9xb krb5kdc[29472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 15:00:02.603888 29707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.598035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:43517 (local address 127.0.0.1:37170)
0926 15:00:02.598104 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.598113 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 15:00:02.598204 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:00:02.598322 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:00:02.598322 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:00:02.598426 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:00:02.598464 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:02.598465 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.599117 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.599118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:02.599609 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:02.599610 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.599997 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.599998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:02.600061 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.600508 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:00:02.600512 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:00:02.601510 (+ 998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:00:02.602951 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:02.602951 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:02.602952 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:02.603098 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:02.603340 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:02.603341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:02.603341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:02.603385 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:02.603601 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:00:02.603602 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:00:02.603641 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 15:00:02.603706 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 15:00:02.603805 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 15:00:02.603909 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.598092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37170 (local address 127.7.37.253:43517)
0926 15:00:02.598133 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.598135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.598144 (+ 9us) server_negotiation.cc:407] Connection header received
0926 15:00:02.598256 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.598257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.598266 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.598314 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.598523 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.599035 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.599673 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.599943 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.599987 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.601582 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:02.601583 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:02.601583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:02.601588 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:02.602887 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.603197 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.603198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.603198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.603267 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.603423 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.603423 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.603424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.603545 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:02.603612 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.603730 (+ 118us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.603823 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 15:00:02.604314 29703 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:00:02.604628 29555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.598173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35834 (local address 127.7.37.254:36259)
0926 15:00:02.598202 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.598203 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.598210 (+ 7us) server_negotiation.cc:407] Connection header received
0926 15:00:02.598225 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.598226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.598234 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.598279 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.598737 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.599170 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.599815 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.599988 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.600047 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.602134 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:02.602135 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:02.602136 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:02.602142 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:02.603492 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.603857 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.603858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.603858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.603915 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.604133 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.604134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.604135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.604258 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:02.604382 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.604479 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.604546 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
Sep 26 15:00:02 dist-test-slave-f9xb krb5kdc[29472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.252@KRBTEST.COM for kudu/127.7.37.252@KRBTEST.COM
Sep 26 15:00:02 dist-test-slave-f9xb krb5kdc[29472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.253@KRBTEST.COM
I20260926 15:00:02.610215 29707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.604545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:44571 (local address 127.0.0.1:40208)
0926 15:00:02.604614 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.604623 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 15:00:02.604661 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:00:02.604866 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:00:02.604867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:00:02.604997 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:00:02.605026 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:02.605027 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.605748 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.605748 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:02.606200 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:02.606201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.606503 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.606503 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:02.606545 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.606895 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:00:02.606898 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:00:02.607890 (+ 992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:00:02.609406 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:02.609406 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:02.609407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:02.609547 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:02.609716 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:02.609717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:02.609717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:02.609763 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:02.609951 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:00:02.609952 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:00:02.609985 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 15:00:02.610041 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 15:00:02.610124 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260926 15:00:02.610265 29706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.604585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40208 (local address 127.7.37.252:44571)
0926 15:00:02.604618 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.604620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.604651 (+ 31us) server_negotiation.cc:407] Connection header received
0926 15:00:02.604781 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.604782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.604792 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.604854 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.605109 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.605691 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.606259 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.606452 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.606501 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.607963 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:02.607964 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:02.607964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:02.607970 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:02.609325 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.609623 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.609624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.609624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.609668 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.609816 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.609817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.609817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.609903 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:02.609951 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.610072 (+ 121us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.610168 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":15}
I20260926 15:00:02.611593 29703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6: Bootstrap starting.
I20260926 15:00:02.612149 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.605312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37180 (local address 127.7.37.253:43517)
0926 15:00:02.605360 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.605362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.605389 (+ 27us) server_negotiation.cc:407] Connection header received
0926 15:00:02.605447 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.605448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.605455 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.605501 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.605907 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.606336 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.607090 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.607298 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.607353 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.609451 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:02.609452 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:02.609453 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:02.609459 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:02.611059 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.611344 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.611345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.611346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.611404 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.611544 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.611544 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.611545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.611662 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:02.611829 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.611946 (+ 117us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.612021 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 15:00:02.612532 29703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6: Neither blocks nor log segments found. Creating new log.
I20260926 15:00:02.612818 29703 log.cc:824] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6: Log is configured to *not* fsync() on all Append() calls
I20260926 15:00:02.613384 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:00:02.613602 29703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6: No bootstrap required, opened a new log
I20260926 15:00:02.614501 29527 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "28286e867f674d08b2c72991cc2d2d93"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35834
I20260926 15:00:02.615075 29703 raft_consensus.cc:348] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28286e867f674d08b2c72991cc2d2d93" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } } peers { permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } } peers { permanent_uuid: "609a800593b8431d9543adc8026f3ae6" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } }
I20260926 15:00:02.615173 29703 raft_consensus.cc:729] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 609a800593b8431d9543adc8026f3ae6, State: Initialized, Role: FOLLOWER
I20260926 15:00:02.615229 29703 consensus_queue.cc:260] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28286e867f674d08b2c72991cc2d2d93" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } } peers { permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } } peers { permanent_uuid: "609a800593b8431d9543adc8026f3ae6" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } }
I20260926 15:00:02.615522 29703 sys_catalog.cc:564] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:00:02.615495 29709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28286e867f674d08b2c72991cc2d2d93" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } } peers { permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } } peers { permanent_uuid: "609a800593b8431d9543adc8026f3ae6" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } } }
I20260926 15:00:02.615599 29709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 15:00:02.617146 29720 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 15:00:02.617233 29720 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 15:00:02.617413 29635 master_runner.cc:186] Error getting master registration for 127.7.37.254:36259: OK, instance_id { permanent_uuid: "28286e867f674d08b2c72991cc2d2d93" instance_seqno: 1790434802302132 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 15:00:02.620198 29635 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:00:02.642135 29553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } attempt: 2
Sep 26 15:00:02 dist-test-slave-f9xb krb5kdc[29472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for kudu/127.7.37.252@KRBTEST.COM
I20260926 15:00:02.649081 29706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.642497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40214 (local address 127.7.37.252:44571)
0926 15:00:02.642591 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.642592 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.642601 (+ 9us) server_negotiation.cc:407] Connection header received
0926 15:00:02.642624 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.642624 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.642635 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.642687 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.642968 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.643678 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.644370 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.644575 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.644635 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.646386 (+ 1751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:02.646388 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:02.646388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:02.646395 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:02.647945 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.648317 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.648318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.648319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.648374 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.648549 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.648550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.648550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.648721 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:02.648785 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.648923 (+ 138us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.648993 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 15:00:02.649086 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.642433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:44571 (local address 127.0.0.1:40214)
0926 15:00:02.642489 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.642500 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 15:00:02.642542 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:00:02.642692 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:00:02.642693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:00:02.642848 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:00:02.642892 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:02.642894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.643773 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.643775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:02.644289 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:02.644291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.644648 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.644648 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:02.644701 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.645129 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:00:02.645133 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:00:02.646282 (+ 1149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:00:02.648020 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:02.648021 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:02.648021 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:02.648248 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:02.648432 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:02.648433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:02.648434 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:02.648493 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:02.648780 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:00:02.648781 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:00:02.648824 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 15:00:02.648896 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 15:00:02.648981 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 15:00:02.650600 29553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93: Bootstrap starting.
I20260926 15:00:02.651278 29553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93: Neither blocks nor log segments found. Creating new log.
I20260926 15:00:02.651582 29553 log.cc:824] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93: Log is configured to *not* fsync() on all Append() calls
I20260926 15:00:02.652446 29553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93: No bootstrap required, opened a new log
I20260926 15:00:02.653759 29553 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28286e867f674d08b2c72991cc2d2d93" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } } peers { permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } } peers { permanent_uuid: "609a800593b8431d9543adc8026f3ae6" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } }
I20260926 15:00:02.653899 29553 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28286e867f674d08b2c72991cc2d2d93, State: Initialized, Role: FOLLOWER
I20260926 15:00:02.653982 29553 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28286e867f674d08b2c72991cc2d2d93" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } } peers { permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } } peers { permanent_uuid: "609a800593b8431d9543adc8026f3ae6" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } }
I20260926 15:00:02.654275 29553 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:00:02.654233 29723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28286e867f674d08b2c72991cc2d2d93" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } } peers { permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } } peers { permanent_uuid: "609a800593b8431d9543adc8026f3ae6" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } } }
I20260926 15:00:02.654345 29723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:00:02.654377 29527 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 15:00:02.654474 29527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:00:02.655174 29527 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28286e867f674d08b2c72991cc2d2d93" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } } peers { permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } } peers { permanent_uuid: "609a800593b8431d9543adc8026f3ae6" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } }
I20260926 15:00:02.655987 29527 leader_election.cc:288] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [CANDIDATE]: Term 1 election: Requested vote from peers 6ef39407bb3c4f5c82d7cc9d31194427 (127.7.37.253:43517), 609a800593b8431d9543adc8026f3ae6 (127.7.37.252:44571)
W20260926 15:00:02.656138 29734 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 15:00:02.656210 29734 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 15:00:02.656232 29677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28286e867f674d08b2c72991cc2d2d93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "609a800593b8431d9543adc8026f3ae6"
I20260926 15:00:02.656354 29677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:00:02.657174 29677 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28286e867f674d08b2c72991cc2d2d93 in term 1.
I20260926 15:00:02.658214 29502 leader_election.cc:302] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28286e867f674d08b2c72991cc2d2d93, 609a800593b8431d9543adc8026f3ae6; no voters:
I20260926 15:00:02.658324 29723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:00:02.658452 29723 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [term 1 LEADER]: Becoming Leader. State: Replica: 28286e867f674d08b2c72991cc2d2d93, State: Running, Role: LEADER
I20260926 15:00:02.658527 29723 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28286e867f674d08b2c72991cc2d2d93" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } } peers { permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } } peers { permanent_uuid: "609a800593b8431d9543adc8026f3ae6" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } }
I20260926 15:00:02.658913 29723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28286e867f674d08b2c72991cc2d2d93. Latest consensus state: current_term: 1 leader_uuid: "28286e867f674d08b2c72991cc2d2d93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28286e867f674d08b2c72991cc2d2d93" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } } peers { permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } } peers { permanent_uuid: "609a800593b8431d9543adc8026f3ae6" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } } }
I20260926 15:00:02.658979 29723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [sys.catalog]: This master's current role is: LEADER
I20260926 15:00:02.659235 29485 master_runner.cc:186] Error getting master registration for 127.7.37.253:43517: OK, instance_id { permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" instance_seqno: 1790434802441600 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 15:00:02.661537 29485 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:00:02.662189 29744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:00:02.662379 29744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:00:02.664394 29677 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6 [term 1 FOLLOWER]: Refusing update from remote peer 28286e867f674d08b2c72991cc2d2d93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:00:02.664899 29723 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "609a800593b8431d9543adc8026f3ae6" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:00:02.666290 29709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28286e867f674d08b2c72991cc2d2d93. Latest consensus state: current_term: 1 leader_uuid: "28286e867f674d08b2c72991cc2d2d93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28286e867f674d08b2c72991cc2d2d93" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } } peers { permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } } peers { permanent_uuid: "609a800593b8431d9543adc8026f3ae6" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } } }
I20260926 15:00:02.666379 29709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:00:02.668581 29723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28286e867f674d08b2c72991cc2d2d93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28286e867f674d08b2c72991cc2d2d93" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } } peers { permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } } peers { permanent_uuid: "609a800593b8431d9543adc8026f3ae6" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } } }
I20260926 15:00:02.668663 29723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [sys.catalog]: This master's current role is: LEADER
I20260926 15:00:02.669121 29709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "28286e867f674d08b2c72991cc2d2d93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28286e867f674d08b2c72991cc2d2d93" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } } peers { permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } } peers { permanent_uuid: "609a800593b8431d9543adc8026f3ae6" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } } }
I20260926 15:00:02.669214 29709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:00:02.670123 29746 mvcc.cc:205] Tried to move back new op lower bound from 7333620951709257728 to 7333620951689977856. Current Snapshot: MvccSnapshot[applied={T|T < 7333620951709257728}]
I20260926 15:00:02.670789 29747 mvcc.cc:205] Tried to move back new op lower bound from 7333620951709257728 to 7333620951689977856. Current Snapshot: MvccSnapshot[applied={T|T < 7333620951709257728}]
I20260926 15:00:02.671172 29706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.658925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40226 (local address 127.7.37.252:44571)
0926 15:00:02.658958 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.658960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.662102 (+ 3142us) server_negotiation.cc:407] Connection header received
0926 15:00:02.663120 (+ 1018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.663121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.663131 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.663158 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.663855 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.664535 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.666201 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.666489 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.666523 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.667956 (+ 1433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:02.667958 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:02.667958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:02.667964 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:02.669931 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.670340 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.670341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.670342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.670392 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.670579 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.670580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.670580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.670709 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:02.670783 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.671022 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.671092 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 15:00:02.671972 29744 catalog_manager.cc:1347] Generated new cluster ID: 122cdc7efe9f4a11882f17bc117423a7
I20260926 15:00:02.672019 29744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:00:02.671984 29628 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } attempt: 3
Sep 26 15:00:02 dist-test-slave-f9xb krb5kdc[29472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.253@KRBTEST.COM for kudu/127.7.37.252@KRBTEST.COM
I20260926 15:00:02.677649 29555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.659153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35840 (local address 127.7.37.254:36259)
0926 15:00:02.659186 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.659188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.671051 (+ 11863us) server_negotiation.cc:407] Connection header received
0926 15:00:02.671154 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.671155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.671169 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.671195 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.671776 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.672435 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.673346 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.673620 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.673663 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.674817 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:02.674818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:02.674818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:02.674824 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:02.676361 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.676764 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.676765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.676765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.676815 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.677010 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.677011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.677012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.677176 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:02.677317 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.677503 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.677563 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 15:00:02.680215 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.658364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37188 (local address 127.7.37.253:43517)
0926 15:00:02.658633 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.658635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.659431 (+ 796us) server_negotiation.cc:407] Connection header received
0926 15:00:02.659522 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.659523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.659531 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.659558 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.660305 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.661032 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.665515 (+ 4483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.665837 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.665876 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.675819 (+ 9943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:02.675821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:02.675822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:02.675828 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:02.678356 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.679299 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.679300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.679301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.679363 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.679593 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.679595 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.679595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.679721 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:02.679841 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.680052 (+ 211us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.680123 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248}
I20260926 15:00:02.682368 29630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.672278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:44571 (local address 127.0.0.1:40242)
0926 15:00:02.672393 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:02.672402 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 15:00:02.672440 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:00:02.672742 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:00:02.672743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:00:02.672921 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:00:02.672961 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:02.672963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.673847 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.673848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:02.674625 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:02.674627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.675155 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.675157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:02.675228 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.675543 (+ 315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:00:02.675547 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:00:02.677682 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:00:02.681033 (+ 3351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:02.681035 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:02.681059 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:02.681260 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:02.681577 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:02.681578 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:02.681578 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:02.681650 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:02.682098 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:00:02.682099 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:00:02.682141 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 15:00:02.682190 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 15:00:02.682277 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260926 15:00:02.683166 29706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.672532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40242 (local address 127.7.37.252:44571)
0926 15:00:02.672594 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.672596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.672603 (+ 7us) server_negotiation.cc:407] Connection header received
0926 15:00:02.672634 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.672635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.672643 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.672691 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.673068 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.673732 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.674737 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.675025 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.675093 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.678427 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:02.678429 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:02.678429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:02.678435 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:02.680923 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.681391 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.681392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.681392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.681474 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:02.681735 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:02.681736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:02.681736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:02.681973 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:02.682769 (+ 796us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.682843 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.682979 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 15:00:02.685137 29628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427: Bootstrap starting.
I20260926 15:00:02.685511 29744 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:00:02.685802 29628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427: Neither blocks nor log segments found. Creating new log.
I20260926 15:00:02.686100 29628 log.cc:824] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427: Log is configured to *not* fsync() on all Append() calls
I20260926 15:00:02.686151 29744 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:00:02.696338 29628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427: No bootstrap required, opened a new log
I20260926 15:00:02.698297 29628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28286e867f674d08b2c72991cc2d2d93" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } } peers { permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } } peers { permanent_uuid: "609a800593b8431d9543adc8026f3ae6" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } }
I20260926 15:00:02.698418 29628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ef39407bb3c4f5c82d7cc9d31194427, State: Initialized, Role: FOLLOWER
I20260926 15:00:02.698486 29628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28286e867f674d08b2c72991cc2d2d93" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } } peers { permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } } peers { permanent_uuid: "609a800593b8431d9543adc8026f3ae6" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } }
I20260926 15:00:02.698786 29628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:00:02.699942 29601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:00:02.700429 29744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93: Generated new TSK 0
I20260926 15:00:02.700590 29744 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:00:02.700779 29601 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427 [term 1 FOLLOWER]: Refusing update from remote peer 28286e867f674d08b2c72991cc2d2d93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:00:02.701072 29740 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:00:02.701244 29602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28286e867f674d08b2c72991cc2d2d93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6ef39407bb3c4f5c82d7cc9d31194427"
I20260926 15:00:02.701941 29602 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28286e867f674d08b2c72991cc2d2d93 in term 1.
I20260926 15:00:02.703692 29762 mvcc.cc:205] Tried to move back new op lower bound from 7333620951709257728 to 7333620951689977856. Current Snapshot: MvccSnapshot[applied={T|T < 7333620951709257728}]
I20260926 15:00:02.704147 29761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28286e867f674d08b2c72991cc2d2d93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28286e867f674d08b2c72991cc2d2d93" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } } peers { permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } } peers { permanent_uuid: "609a800593b8431d9543adc8026f3ae6" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } } }
I20260926 15:00:02.704241 29761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:00:02.704298 29761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28286e867f674d08b2c72991cc2d2d93. Latest consensus state: current_term: 1 leader_uuid: "28286e867f674d08b2c72991cc2d2d93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28286e867f674d08b2c72991cc2d2d93" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } } peers { permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } } peers { permanent_uuid: "609a800593b8431d9543adc8026f3ae6" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } } }
I20260926 15:00:02.704339 29761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:00:02.704365 29761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "28286e867f674d08b2c72991cc2d2d93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28286e867f674d08b2c72991cc2d2d93" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } } peers { permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } } peers { permanent_uuid: "609a800593b8431d9543adc8026f3ae6" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } } }
I20260926 15:00:02.704397 29761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:00:02.706030 29771 catalog_manager.cc:1259] Loaded cluster ID: 122cdc7efe9f4a11882f17bc117423a7
I20260926 15:00:02.706123 29771 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427: loading cluster ID for follower catalog manager: success
I20260926 15:00:02.711227 29771 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427: acquiring CA information for follower catalog manager: success
I20260926 15:00:02.711643 29771 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6ef39407bb3c4f5c82d7cc9d31194427: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 15:00:02.728065 29632 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:02.726557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37196 (local address 127.7.37.253:43517)
0926 15:00:02.726609 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.726611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.727443 (+ 832us) server_negotiation.cc:407] Connection header received
0926 15:00:02.727534 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.727535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.727545 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.727588 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.728006 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37196: BlockingRecv error: recv got EOF from 127.0.0.1:37196 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260926 15:00:02.728725 29706 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:02.726931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40244 (local address 127.7.37.252:44571)
0926 15:00:02.726965 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.726967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.728276 (+ 1309us) server_negotiation.cc:407] Connection header received
0926 15:00:02.728358 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.728359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.728368 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.728392 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.728671 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40244: BlockingRecv error: recv got EOF from 127.0.0.1:40244 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260926 15:00:02.729362 29555 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:02.727225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35844 (local address 127.7.37.254:36259)
0926 15:00:02.727266 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.727268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.728916 (+ 1648us) server_negotiation.cc:407] Connection header received
0926 15:00:02.729001 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.729002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.729011 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.729037 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.729303 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35844: BlockingRecv error: recv got EOF from 127.0.0.1:35844 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 15:00:02.734135 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.730849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37200 (local address 127.7.37.253:43517)
0926 15:00:02.730879 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.730881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.731060 (+ 179us) server_negotiation.cc:407] Connection header received
0926 15:00:02.731149 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.731150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.731183 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.731220 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:02.731394 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.732195 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.733279 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.733560 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.733610 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.733804 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:02.733916 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:02.733959 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.734056 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.734073 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260926 15:00:02.737141 29706 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:02.736394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40258 (local address 127.7.37.252:44571)
0926 15:00:02.736428 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.736430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.736668 (+ 238us) server_negotiation.cc:407] Connection header received
0926 15:00:02.736754 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.736755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.736770 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.736799 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.737096 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40258: BlockingRecv error: recv got EOF from 127.0.0.1:40258 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 15:00:02.742202 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.739239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37210 (local address 127.7.37.253:43517)
0926 15:00:02.739269 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.739271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.739492 (+ 221us) server_negotiation.cc:407] Connection header received
0926 15:00:02.739590 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.739591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.739599 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.739623 (+ 24us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:02.739813 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.740464 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.741441 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.741738 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.741775 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.741962 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:02.741990 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:02.742014 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.742125 (+ 111us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.742143 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 15:00:02.745076 29706 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:02.744374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40270 (local address 127.7.37.252:44571)
0926 15:00:02.744406 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.744408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.744606 (+ 198us) server_negotiation.cc:407] Connection header received
0926 15:00:02.744699 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.744700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.744711 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.744738 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.745033 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40270: BlockingRecv error: recv got EOF from 127.0.0.1:40270 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 15:00:02.751044 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.748091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37216 (local address 127.7.37.253:43517)
0926 15:00:02.748123 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.748124 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.748345 (+ 221us) server_negotiation.cc:407] Connection header received
0926 15:00:02.748429 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.748430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.748437 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.748462 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:02.748626 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.749288 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.750329 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.750612 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.750652 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.750824 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:02.750854 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:02.750878 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.750971 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.750987 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 15:00:02.753757 29706 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:02.753077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40272 (local address 127.7.37.252:44571)
0926 15:00:02.753110 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.753111 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.753284 (+ 173us) server_negotiation.cc:407] Connection header received
0926 15:00:02.753368 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.753369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.753379 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.753405 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.753715 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40272: BlockingRecv error: recv got EOF from 127.0.0.1:40272 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 15:00:02.761790 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.758716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37226 (local address 127.7.37.253:43517)
0926 15:00:02.758747 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.758748 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.758953 (+ 205us) server_negotiation.cc:407] Connection header received
0926 15:00:02.759029 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.759030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.759037 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.759060 (+ 23us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:02.759215 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.759934 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.761056 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.761333 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.761369 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.761542 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:02.761573 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:02.761595 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.761706 (+ 111us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.761723 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 15:00:02.764511 29706 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:02.763849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40274 (local address 127.7.37.252:44571)
0926 15:00:02.763889 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.763891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.764059 (+ 168us) server_negotiation.cc:407] Connection header received
0926 15:00:02.764140 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.764141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.764151 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.764175 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.764464 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40274: BlockingRecv error: recv got EOF from 127.0.0.1:40274 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 15:00:02.776203 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.773455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37240 (local address 127.7.37.253:43517)
0926 15:00:02.773486 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.773488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.773851 (+ 363us) server_negotiation.cc:407] Connection header received
0926 15:00:02.773940 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.773941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.773950 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.773975 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:02.774123 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.774803 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.775528 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.775698 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.775781 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.775917 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:02.775944 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:02.775982 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.776119 (+ 137us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.776137 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260926 15:00:02.778725 29706 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:02.778228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40290 (local address 127.7.37.252:44571)
0926 15:00:02.778263 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.778265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.778328 (+ 63us) server_negotiation.cc:407] Connection header received
0926 15:00:02.778376 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.778377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.778386 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.778425 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.778670 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40290: BlockingRecv error: recv got EOF from 127.0.0.1:40290 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 15:00:02.797772 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.795590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37254 (local address 127.7.37.253:43517)
0926 15:00:02.795627 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.795629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.795727 (+ 98us) server_negotiation.cc:407] Connection header received
0926 15:00:02.795830 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.795831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.795840 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.795886 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:02.795994 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.796483 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.797206 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.797384 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.797437 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.797558 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:02.797586 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:02.797624 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.797682 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.797698 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 15:00:02.800021 29706 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:02.799306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40296 (local address 127.7.37.252:44571)
0926 15:00:02.799356 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.799358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.799479 (+ 121us) server_negotiation.cc:407] Connection header received
0926 15:00:02.799597 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.799598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.799610 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.799663 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.799967 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40296: BlockingRecv error: recv got EOF from 127.0.0.1:40296 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 15:00:02.835160 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.832913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37270 (local address 127.7.37.253:43517)
0926 15:00:02.832956 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.832958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.833075 (+ 117us) server_negotiation.cc:407] Connection header received
0926 15:00:02.833134 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.833135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.833145 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.833191 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:02.833301 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.833804 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.834570 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.834762 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.834815 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.834928 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:02.834961 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:02.835010 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.835059 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.835076 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260926 15:00:02.837380 29706 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:02.836694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40300 (local address 127.7.37.252:44571)
0926 15:00:02.836737 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.836739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.836928 (+ 189us) server_negotiation.cc:407] Connection header received
0926 15:00:02.836982 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.836983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.836993 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.837036 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.837327 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40300: BlockingRecv error: recv got EOF from 127.0.0.1:40300 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 15:00:02.904644 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:02.902424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37276 (local address 127.7.37.253:43517)
0926 15:00:02.902500 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.902502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.902637 (+ 135us) server_negotiation.cc:407] Connection header received
0926 15:00:02.902667 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.902668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.902677 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.902727 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:02.902857 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.903329 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.904049 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:02.904228 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:02.904272 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:02.904445 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:02.904468 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:02.904498 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:02.904557 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 15:00:02.904569 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260926 15:00:02.906693 29706 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:02.906188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40316 (local address 127.7.37.252:44571)
0926 15:00:02.906264 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:02.906265 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:02.906274 (+ 9us) server_negotiation.cc:407] Connection header received
0926 15:00:02.906298 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:02.906298 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:02.906310 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:02.906351 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:02.906635 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40316: BlockingRecv error: recv got EOF from 127.0.0.1:40316 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 15:00:03.037883 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:03.035833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37288 (local address 127.7.37.253:43517)
0926 15:00:03.035894 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:03.035896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:03.035934 (+ 38us) server_negotiation.cc:407] Connection header received
0926 15:00:03.035998 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:03.035999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:03.036007 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:03.036065 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:03.036184 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:03.036658 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:03.037365 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:03.037548 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:03.037603 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:03.037712 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:03.037736 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:03.037767 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:03.037810 (+ 43us) server_negotiation.cc:299] Negotiation successful
0926 15:00:03.037823 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260926 15:00:03.040076 29706 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:03.039511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40330 (local address 127.7.37.252:44571)
0926 15:00:03.039592 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:03.039594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:03.039603 (+ 9us) server_negotiation.cc:407] Connection header received
0926 15:00:03.039627 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:03.039628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:03.039636 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:03.039697 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:03.040021 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40330: BlockingRecv error: recv got EOF from 127.0.0.1:40330 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 15:00:03.254722 29872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28286e867f674d08b2c72991cc2d2d93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28286e867f674d08b2c72991cc2d2d93" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 36259 } } peers { permanent_uuid: "6ef39407bb3c4f5c82d7cc9d31194427" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43517 } } peers { permanent_uuid: "609a800593b8431d9543adc8026f3ae6" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 44571 } } }
I20260926 15:00:03.254873 29872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28286e867f674d08b2c72991cc2d2d93 [sys.catalog]: This master's current role is: LEADER
I20260926 15:00:03.299449 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:03.297217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37300 (local address 127.7.37.253:43517)
0926 15:00:03.297287 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:03.297289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:03.297317 (+ 28us) server_negotiation.cc:407] Connection header received
0926 15:00:03.297391 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:03.297392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:03.297402 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:03.297457 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:03.297576 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:03.298062 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:03.298849 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:03.299032 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:03.299094 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:03.299234 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:03.299262 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:03.299300 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:03.299339 (+ 39us) server_negotiation.cc:299] Negotiation successful
0926 15:00:03.299354 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260926 15:00:03.301661 29706 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:03.301114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40340 (local address 127.7.37.252:44571)
0926 15:00:03.301183 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:03.301185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:03.301193 (+ 8us) server_negotiation.cc:407] Connection header received
0926 15:00:03.301218 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:03.301218 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:03.301230 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:03.301284 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:03.301600 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40340: BlockingRecv error: recv got EOF from 127.0.0.1:40340 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 15:00:03.617650 29720 catalog_manager.cc:1259] Loaded cluster ID: 122cdc7efe9f4a11882f17bc117423a7
I20260926 15:00:03.617713 29720 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6: loading cluster ID for follower catalog manager: success
I20260926 15:00:03.618489 29720 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6: acquiring CA information for follower catalog manager: success
I20260926 15:00:03.618777 29720 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 609a800593b8431d9543adc8026f3ae6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:00:03.816987 29891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:03.814697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37306 (local address 127.7.37.253:43517)
0926 15:00:03.814783 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:03.814785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:03.814794 (+ 9us) server_negotiation.cc:407] Connection header received
0926 15:00:03.814864 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:03.814864 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:03.814874 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:03.814930 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:03.815034 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:03.815520 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:03.816387 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:03.816573 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:03.816624 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:03.816724 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:03.816755 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:03.816797 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:03.816884 (+ 87us) server_negotiation.cc:299] Negotiation successful
0926 15:00:03.816904 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70,"thread_start_us":46,"threads_started":1}
I20260926 15:00:03.820596 29897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:03.818519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40342 (local address 127.7.37.252:44571)
0926 15:00:03.818624 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:03.818626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:03.818633 (+ 7us) server_negotiation.cc:407] Connection header received
0926 15:00:03.818653 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:03.818653 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:03.818695 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:03.818760 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:03.818824 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:03.819293 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:03.819981 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:03.820170 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:03.820245 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:03.820339 (+ 94us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:03.820440 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:03.820491 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:03.820506 (+ 15us) server_negotiation.cc:299] Negotiation successful
0926 15:00:03.820522 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":50,"threads_started":1}
I20260926 15:00:03.824940 29891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:03.822123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37310 (local address 127.7.37.253:43517)
0926 15:00:03.822154 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:03.822156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:03.822253 (+ 97us) server_negotiation.cc:407] Connection header received
0926 15:00:03.822338 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:03.822339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:03.822346 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:03.822372 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:03.822516 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:03.823187 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:03.824257 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:03.824535 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:03.824588 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:03.824730 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:03.824760 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:03.824803 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:03.824859 (+ 56us) server_negotiation.cc:299] Negotiation successful
0926 15:00:03.824878 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260926 15:00:03.825188 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:03.822034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35856 (local address 127.7.37.254:36259)
0926 15:00:03.822147 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:03.822150 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:03.822296 (+ 146us) server_negotiation.cc:407] Connection header received
0926 15:00:03.822345 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:03.822346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:03.822391 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:03.822459 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:03.822617 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:03.823343 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:03.824280 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:03.824491 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:03.824541 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:03.824753 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:03.824854 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:03.824906 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:03.825087 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 15:00:03.825112 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":51,"threads_started":1}
I20260926 15:00:03.825419 29897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:03.822088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40358 (local address 127.7.37.252:44571)
0926 15:00:03.822163 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:03.822165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:03.822279 (+ 114us) server_negotiation.cc:407] Connection header received
0926 15:00:03.822333 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:03.822333 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:03.822341 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:03.822408 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:03.823301 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:03.823785 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:03.824646 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:03.824909 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:03.824959 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:03.825130 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:03.825159 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:03.825215 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:03.825339 (+ 124us) server_negotiation.cc:299] Negotiation successful
0926 15:00:03.825358 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 15:00:03.826779 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 29485
I20260926 15:00:03.831008 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 29560
I20260926 15:00:03.834740 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 29635
2026-09-26T15:00:03Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2696 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:00:03 dist-test-slave-f9xb krb5kdc[29908](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:00:03 dist-test-slave-f9xb krb5kdc[29908](info): set up 2 sockets
Sep 26 15:00:03 dist-test-slave-f9xb krb5kdc[29908](info): commencing operation
krb5kdc: starting...
W20260926 15:00:04.767612 7319 mini_kdc.cc:121] Time spent starting KDC: real 0.915s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/test-user.keytab.
Sep 26 15:00:04 dist-test-slave-f9xb krb5kdc[29908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:00:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:00:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.37.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
Entry for principal kudu/127.7.37.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab.
I20260926 15:00:04.871992 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:32829
--webserver_interface=127.7.37.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:43361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:32829,127.7.37.253:43841,127.7.37.252:40427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:00:04.947602 29921 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 15:00:04.947805 29921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:00:04.947821 29921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:00:04.947839 29921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:00:04.948923 29921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:00:04.948961 29921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:00:04.948972 29921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:00:04.948983 29921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:00:04.949007 29921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:00:04.950690 29921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:43361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:32829,127.7.37.253:43841,127.7.37.252:40427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.254.keytab
--principal=kudu/127.7.37.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.254:32829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.37.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 15:00:04.950928 29921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:00:04.951156 29921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:00:04.955616 29926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:00:04.955662 29929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:00:04.955819 29927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:00:04.955799 29921 server_base.cc:1034] running on GCE node
I20260926 15:00:04.956063 29921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:00:04.956355 29921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:00:04.957506 29921 hybrid_clock.cc:648] HybridClock initialized: now 1790434804957484 us; error 38 us; skew 500 ppm
Sep 26 15:00:04 dist-test-slave-f9xb krb5kdc[29908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434804, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:00:04.959769 29921 init.cc:377] Logged in from keytab as kudu/127.7.37.254@KRBTEST.COM (short username kudu)
I20260926 15:00:04.960170 29921 webserver.cc:460] Webserver started at http://127.7.37.254:36933/ using document root <none> and password file <none>
I20260926 15:00:04.960332 29921 fs_manager.cc:359] Metadata directory not provided
I20260926 15:00:04.960367 29921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:00:04.960450 29921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:00:04.961339 29921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-0/data/instance:
uuid: "fb76135da89943639740f1afd56583a4"
format_stamp: "Formatted at 2026-09-26 15:00:04 on dist-test-slave-f9xb"
server_key: "3dfb9301077dcb87f1b1daee1352b7be"
server_key_iv: "56b4ad55a856d0ee517e672e618ec3f3"
server_key_version: "encryptionkey@0"
I20260926 15:00:04.961607 29921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-0/wal/instance:
uuid: "fb76135da89943639740f1afd56583a4"
format_stamp: "Formatted at 2026-09-26 15:00:04 on dist-test-slave-f9xb"
server_key: "3dfb9301077dcb87f1b1daee1352b7be"
server_key_iv: "56b4ad55a856d0ee517e672e618ec3f3"
server_key_version: "encryptionkey@0"
I20260926 15:00:04.962723 29921 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 15:00:04.963373 29936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:00:04.963582 29921 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 15:00:04.963663 29921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-0/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-0/wal
uuid: "fb76135da89943639740f1afd56583a4"
format_stamp: "Formatted at 2026-09-26 15:00:04 on dist-test-slave-f9xb"
server_key: "3dfb9301077dcb87f1b1daee1352b7be"
server_key_iv: "56b4ad55a856d0ee517e672e618ec3f3"
server_key_version: "encryptionkey@0"
I20260926 15:00:04.963719 29921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:00:04.992432 29921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:00:04.994436 29921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:00:04.994539 29921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:00:04.998140 29921 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.254:32829
I20260926 15:00:04.998173 29988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.254:32829 every 8 connection(s)
I20260926 15:00:04.998481 29921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-0/data/info.pb
I20260926 15:00:04.999442 29989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:00:05.000933 29989 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:00:05.005891 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 29921
I20260926 15:00:05.005966 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-0/wal/instance
I20260926 15:00:05.006137 7319 external_mini_cluster.cc:1428] Setting key 17d1b92b2d57e1addb9bf0c439789d94
Sep 26 15:00:05 dist-test-slave-f9xb krb5kdc[29908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434804, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 15:00:05.009548 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.001544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:32829 (local address 127.0.0.1:34314)
0926 15:00:05.001700 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.001740 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 15:00:05.002162 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:00:05.002392 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:00:05.002395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:00:05.002612 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:00:05.002766 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:05.002770 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.003680 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.003681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:05.004349 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:05.004351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.004814 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.004815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:05.004877 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.005280 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:00:05.005288 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:00:05.006612 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:00:05.008235 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:05.008241 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:05.008243 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:05.008419 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:05.008652 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:05.008652 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:05.008653 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:05.008720 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:05.009077 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:00:05.009079 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:00:05.009148 (+ 69us) client_negotiation.cc:769] Sending connection context
0926 15:00:05.009239 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 15:00:05.009376 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
I20260926 15:00:05.009558 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.001588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34314 (local address 127.7.37.254:32829)
0926 15:00:05.001738 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.001747 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.001826 (+ 79us) server_negotiation.cc:407] Connection header received
0926 15:00:05.002276 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.002277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.002300 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.002376 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.002834 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.003611 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.004410 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.004702 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.004776 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.006691 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:05.006698 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:05.006698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:05.006704 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:05.008168 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.008490 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.008492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.008493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.008595 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.008773 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.008773 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.008774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.009019 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:05.009094 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:05.009280 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 15:00:05.009445 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260926 15:00:05.010432 29989 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:00:05.010746 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.010633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:43841 (local address 127.0.0.1:45938)
0926 15:00:05.010680 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.010723 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 15:00:05.010845 29939 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43841: connect: Connection refused (error 111)
I20260926 15:00:05.011224 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.011120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:43841 (local address 127.0.0.1:45952)
0926 15:00:05.011171 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.011200 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260926 15:00:05.011339 29989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.253:43841: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43841: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.7.37.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.253.keytab.
Entry for principal kudu/127.7.37.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.253.keytab.
I20260926 15:00:05.029953 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.253:43841
--webserver_interface=127.7.37.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.253.keytab
--principal=kudu/127.7.37.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:43361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:32829,127.7.37.253:43841,127.7.37.252:40427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:00:05.060544 29989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } attempt: 1
I20260926 15:00:05.060992 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.060841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:43841 (local address 127.0.0.1:45960)
0926 15:00:05.060907 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.060953 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 15:00:05.061337 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.061251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:43841 (local address 127.0.0.1:45976)
0926 15:00:05.061294 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.061319 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260926 15:00:05.061434 29989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.253:43841: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43841: connect: Connection refused (error 111)
W20260926 15:00:05.109386 29996 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 15:00:05.109576 29996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:00:05.109591 29996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:00:05.109609 29996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:00:05.110721 29996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:00:05.110760 29996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:00:05.110772 29996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:00:05.110783 29996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:00:05.110805 29996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:00:05.112254 29996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:43361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:32829,127.7.37.253:43841,127.7.37.252:40427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.253.keytab
--principal=kudu/127.7.37.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.253:43841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.37.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 15:00:05.112466 29996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:00:05.112661 29996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:00:05.117412 30002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:00:05.117432 30004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:00:05.117542 30001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:00:05.117674 29996 server_base.cc:1034] running on GCE node
I20260926 15:00:05.117877 29996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:00:05.118098 29996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:00:05.119292 29996 hybrid_clock.cc:648] HybridClock initialized: now 1790434805119278 us; error 27 us; skew 500 ppm
Sep 26 15:00:05 dist-test-slave-f9xb krb5kdc[29908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434805, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:00:05.121695 29996 init.cc:377] Logged in from keytab as kudu/127.7.37.253@KRBTEST.COM (short username kudu)
I20260926 15:00:05.122164 29996 webserver.cc:460] Webserver started at http://127.7.37.253:36391/ using document root <none> and password file <none>
I20260926 15:00:05.122365 29996 fs_manager.cc:359] Metadata directory not provided
I20260926 15:00:05.122414 29996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:00:05.122545 29996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:00:05.123615 29996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-1/data/instance:
uuid: "cfd605b154884ef9aba07963570a39d3"
format_stamp: "Formatted at 2026-09-26 15:00:05 on dist-test-slave-f9xb"
server_key: "486f86394ca8d8c7be76afe28be07158"
server_key_iv: "e8e380a862e574f18c1035631e2882c1"
server_key_version: "encryptionkey@0"
I20260926 15:00:05.123981 29996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-1/wal/instance:
uuid: "cfd605b154884ef9aba07963570a39d3"
format_stamp: "Formatted at 2026-09-26 15:00:05 on dist-test-slave-f9xb"
server_key: "486f86394ca8d8c7be76afe28be07158"
server_key_iv: "e8e380a862e574f18c1035631e2882c1"
server_key_version: "encryptionkey@0"
I20260926 15:00:05.125221 29996 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 15:00:05.125913 30011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:00:05.126081 29996 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 15:00:05.126164 29996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-1/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-1/wal
uuid: "cfd605b154884ef9aba07963570a39d3"
format_stamp: "Formatted at 2026-09-26 15:00:05 on dist-test-slave-f9xb"
server_key: "486f86394ca8d8c7be76afe28be07158"
server_key_iv: "e8e380a862e574f18c1035631e2882c1"
server_key_version: "encryptionkey@0"
I20260926 15:00:05.126287 29996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:00:05.129580 29989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } attempt: 2
I20260926 15:00:05.129937 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.129805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:43841 (local address 127.0.0.1:45982)
0926 15:00:05.129866 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.129903 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 15:00:05.130275 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.130182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:43841 (local address 127.0.0.1:45996)
0926 15:00:05.130226 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.130253 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 15:00:05.130366 29989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.253:43841: Network error: Client connection negotiation failed: client connection to 127.7.37.253:43841: connect: Connection refused (error 111)
I20260926 15:00:05.143496 29996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:00:05.145332 29996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:00:05.145437 29996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:00:05.149042 29996 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.253:43841
I20260926 15:00:05.149078 30063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.253:43841 every 8 connection(s)
I20260926 15:00:05.149376 29996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-1/data/info.pb
I20260926 15:00:05.150264 30064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:00:05.151664 30064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:00:05.153893 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 29996
I20260926 15:00:05.153991 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-1/wal/instance
I20260926 15:00:05.154186 7319 external_mini_cluster.cc:1428] Setting key 6245ac136682f2ed945c85c8a1ca5b72
Sep 26 15:00:05 dist-test-slave-f9xb krb5kdc[29908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434805, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.253@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 15:00:05.160960 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.152356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34320 (local address 127.7.37.254:32829)
0926 15:00:05.152414 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.152416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.152590 (+ 174us) server_negotiation.cc:407] Connection header received
0926 15:00:05.153038 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.153039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.153059 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.153123 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.153700 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.154252 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.155592 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.155940 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.156017 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.158155 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:05.158157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:05.158157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:05.158165 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:05.159563 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.159977 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.159978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.159979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.160038 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.160304 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.160305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.160305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.160447 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:05.160524 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:05.160788 (+ 264us) server_negotiation.cc:299] Negotiation successful
0926 15:00:05.160868 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 15:00:05.160989 30066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.152309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:32829 (local address 127.0.0.1:34320)
0926 15:00:05.152475 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.152513 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 15:00:05.152959 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:00:05.153203 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:00:05.153206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:00:05.153429 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:00:05.153602 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:05.153606 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.154355 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.154356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:05.155483 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:05.155486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.156032 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.156032 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:05.156132 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.156572 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:00:05.156580 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:00:05.158057 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:00:05.159654 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:05.159660 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:05.159662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:05.159908 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:05.160101 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:05.160101 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:05.160101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:05.160213 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:05.160531 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:00:05.160537 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:00:05.160654 (+ 117us) client_negotiation.cc:769] Sending connection context
0926 15:00:05.160737 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 15:00:05.160843 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260926 15:00:05.161509 30064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.37.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.37.252@KRBTEST.COM" created.
Sep 26 15:00:05 dist-test-slave-f9xb krb5kdc[29908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434805, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.253@KRBTEST.COM for kudu/127.7.37.253@KRBTEST.COM
I20260926 15:00:05.168160 30066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.161744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:43841 (local address 127.0.0.1:46008)
0926 15:00:05.161913 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.161923 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 15:00:05.161971 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:00:05.162088 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:00:05.162089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:00:05.162233 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:00:05.162272 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:05.162274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.163056 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.163057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:05.163537 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:05.163538 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.164018 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.164019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:05.164075 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.164436 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:00:05.164440 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:00:05.165541 (+ 1101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:00:05.167114 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:05.167115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:05.167116 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:05.167316 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:05.167499 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:05.167499 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:05.167500 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:05.167558 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:05.167820 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:00:05.167820 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:00:05.167865 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 15:00:05.167938 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 15:00:05.168033 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260926 15:00:05.168231 30068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.161791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46008 (local address 127.7.37.253:43841)
0926 15:00:05.161932 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.161936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.161969 (+ 33us) server_negotiation.cc:407] Connection header received
0926 15:00:05.162020 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.162023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.162034 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.162089 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.162342 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.162976 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.163606 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.163953 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.164009 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.165605 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:05.165613 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:05.165613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:05.165621 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:05.167027 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.167392 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.167394 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.167394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.167437 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.167617 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.167617 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.167617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.167758 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:05.167826 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:05.167961 (+ 135us) server_negotiation.cc:299] Negotiation successful
0926 15:00:05.168077 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
I20260926 15:00:05.168927 30064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:00:05.169296 30066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.169173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:40427 (local address 127.0.0.1:33078)
0926 15:00:05.169224 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.169272 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:40427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260926 15:00:05.169420 30012 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.37.252:40427: connect: Connection refused (error 111)
I20260926 15:00:05.169818 30066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.169703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:40427 (local address 127.0.0.1:33084)
0926 15:00:05.169751 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.169779 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:40427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 15:00:05.169919 30064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.252:40427: Network error: Client connection negotiation failed: client connection to 127.7.37.252:40427: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.37.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.252.keytab.
Entry for principal kudu/127.7.37.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.252.keytab.
I20260926 15:00:05.178696 7319 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrMzbz4/build/release/bin/kudu
/tmp/dist-test-taskrMzbz4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.252:40427
--webserver_interface=127.7.37.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.252.keytab
--principal=kudu/127.7.37.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.37.212:43361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.37.254:32829,127.7.37.253:43841,127.7.37.252:40427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:00:05.219148 30064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } attempt: 1
I20260926 15:00:05.219615 30066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.219488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:40427 (local address 127.0.0.1:33088)
0926 15:00:05.219547 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.219583 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:40427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 15:00:05.219974 30066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.219890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:40427 (local address 127.0.0.1:33102)
0926 15:00:05.219930 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.219955 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:40427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260926 15:00:05.220064 30064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.252:40427: Network error: Client connection negotiation failed: client connection to 127.7.37.252:40427: connect: Connection refused (error 111)
I20260926 15:00:05.221534 29989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } attempt: 3
Sep 26 15:00:05 dist-test-slave-f9xb krb5kdc[29908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434804, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for kudu/127.7.37.253@KRBTEST.COM
I20260926 15:00:05.228457 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.221799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:43841 (local address 127.0.0.1:46022)
0926 15:00:05.221854 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.221865 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 15:00:05.221910 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:00:05.222056 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:00:05.222057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:00:05.222191 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:00:05.222254 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:05.222256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.223158 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.223159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:05.223680 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:05.223681 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.224178 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.224179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:05.224238 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.224671 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:00:05.224675 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:00:05.225961 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:00:05.227541 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:05.227543 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:05.227543 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:05.227709 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:05.227917 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:05.227917 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:05.227917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:05.227969 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:05.228234 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:00:05.228235 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:00:05.228274 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 15:00:05.228325 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 15:00:05.228395 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260926 15:00:05.228526 30068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.221870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46022 (local address 127.7.37.253:43841)
0926 15:00:05.221927 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.221929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.221958 (+ 29us) server_negotiation.cc:407] Connection header received
0926 15:00:05.221983 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.221984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.221994 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.222044 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.222319 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.223073 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.223777 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.224110 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.224176 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.226062 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:05.226064 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:05.226064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:05.226071 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:05.227463 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.227800 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.227802 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.227802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.227860 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.228026 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.228026 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.228027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.228175 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:05.228236 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:05.228367 (+ 131us) server_negotiation.cc:299] Negotiation successful
0926 15:00:05.228441 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 15:00:05.228834 29989 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:00:05.229115 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.229022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:40427 (local address 127.0.0.1:33116)
0926 15:00:05.229054 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.229087 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:40427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260926 15:00:05.229418 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.229332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:40427 (local address 127.0.0.1:33126)
0926 15:00:05.229375 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.229400 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:40427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 15:00:05.229569 29989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.252:40427: Network error: Client connection negotiation failed: client connection to 127.7.37.252:40427: connect: Connection refused (error 111)
W20260926 15:00:05.256928 30071 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 15:00:05.257112 30071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:00:05.257128 30071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:00:05.257144 30071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:00:05.258239 30071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:00:05.258277 30071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:00:05.258291 30071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:00:05.258301 30071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:00:05.258320 30071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:00:05.259716 30071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.37.212:43361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.37.254:32829,127.7.37.253:43841,127.7.37.252:40427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/krb5kdc/kudu_127.7.37.252.keytab
--principal=kudu/127.7.37.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.37.252:40427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.37.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 13:59:08 UTC on ccycloud.quasar-qegwso.root.comops.site
build id 4674340
I20260926 15:00:05.259941 30071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:00:05.260154 30071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 15:00:05.260718 29989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } attempt: 1
I20260926 15:00:05.261065 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.260929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:40427 (local address 127.0.0.1:33138)
0926 15:00:05.260993 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.261032 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:40427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260926 15:00:05.261443 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.261298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:40427 (local address 127.0.0.1:33148)
0926 15:00:05.261394 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.261418 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.37.252:40427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":57,"threads_started":1}
W20260926 15:00:05.261539 29989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.37.252:40427: Network error: Client connection negotiation failed: client connection to 127.7.37.252:40427: connect: Connection refused (error 111)
W20260926 15:00:05.264649 30077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:00:05.264643 30079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:00:05.264662 30076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:00:05.264902 30071 server_base.cc:1034] running on GCE node
I20260926 15:00:05.265103 30071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:00:05.265337 30071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:00:05.266469 30071 hybrid_clock.cc:648] HybridClock initialized: now 1790434805266456 us; error 27 us; skew 500 ppm
Sep 26 15:00:05 dist-test-slave-f9xb krb5kdc[29908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434805, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:00:05.268785 30071 init.cc:377] Logged in from keytab as kudu/127.7.37.252@KRBTEST.COM (short username kudu)
I20260926 15:00:05.269151 30071 webserver.cc:460] Webserver started at http://127.7.37.252:36239/ using document root <none> and password file <none>
I20260926 15:00:05.269371 30071 fs_manager.cc:359] Metadata directory not provided
I20260926 15:00:05.269416 30071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:00:05.269524 30071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:00:05.270473 30071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-2/data/instance:
uuid: "cbc3d0f76861414d889d7ae54ee8d033"
format_stamp: "Formatted at 2026-09-26 15:00:05 on dist-test-slave-f9xb"
server_key: "938383b9ad88d4e11264288e9360049a"
server_key_iv: "9388e0487fa34649b9691d4300b9891a"
server_key_version: "encryptionkey@0"
I20260926 15:00:05.270772 30071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-2/wal/instance:
uuid: "cbc3d0f76861414d889d7ae54ee8d033"
format_stamp: "Formatted at 2026-09-26 15:00:05 on dist-test-slave-f9xb"
server_key: "938383b9ad88d4e11264288e9360049a"
server_key_iv: "9388e0487fa34649b9691d4300b9891a"
server_key_version: "encryptionkey@0"
I20260926 15:00:05.271956 30071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 15:00:05.272601 30087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:00:05.272758 30071 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 15:00:05.272832 30071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-2/data,/tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-2/wal
uuid: "cbc3d0f76861414d889d7ae54ee8d033"
format_stamp: "Formatted at 2026-09-26 15:00:05 on dist-test-slave-f9xb"
server_key: "938383b9ad88d4e11264288e9360049a"
server_key_iv: "9388e0487fa34649b9691d4300b9891a"
server_key_version: "encryptionkey@0"
I20260926 15:00:05.272889 30071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:00:05.279515 30071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:00:05.281431 30071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:00:05.281548 30071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:00:05.284951 30071 rpc_server.cc:307] RPC server started. Bound to: 127.7.37.252:40427
I20260926 15:00:05.284986 30139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.37.252:40427 every 8 connection(s)
I20260926 15:00:05.285274 30071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-2/data/info.pb
I20260926 15:00:05.286131 30140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:00:05.287477 30140 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:00:05.288218 30064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } attempt: 2
I20260926 15:00:05.292431 7319 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrMzbz4/build/release/bin/kudu as pid 30071
I20260926 15:00:05.292526 7319 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrMzbz4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434555014799-7319-0/minicluster-data/master-2/wal/instance
Sep 26 15:00:05 dist-test-slave-f9xb krb5kdc[29908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434805, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.252@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 15:00:05.292721 7319 external_mini_cluster.cc:1428] Setting key b9a9a99387a2fecb384e02a4b94a2eb0
Sep 26 15:00:05 dist-test-slave-f9xb krb5kdc[29908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434805, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.253@KRBTEST.COM for kudu/127.7.37.252@KRBTEST.COM
I20260926 15:00:05.296010 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.288118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34326 (local address 127.7.37.254:32829)
0926 15:00:05.288194 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.288196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.288367 (+ 171us) server_negotiation.cc:407] Connection header received
0926 15:00:05.288912 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.288913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.288924 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.288998 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.289539 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.290024 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.290849 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.291045 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.291098 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.293089 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:05.293091 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:05.293091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:05.293097 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:05.294508 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.294980 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.294981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.294981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.295038 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.295267 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.295268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.295269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.295374 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:05.295427 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:05.295839 (+ 412us) server_negotiation.cc:299] Negotiation successful
0926 15:00:05.295915 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 15:00:05.296082 30142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.288067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.254:32829 (local address 127.0.0.1:34326)
0926 15:00:05.288245 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.288286 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 15:00:05.288800 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:00:05.289018 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:00:05.289021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:00:05.289270 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:00:05.289407 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:05.289411 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.290099 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.290100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:05.290764 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:05.290766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.291130 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.291131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:05.291227 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.291621 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:00:05.291628 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:00:05.292990 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:00:05.294587 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:05.294593 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:05.294595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:05.294906 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:05.295099 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:05.295099 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:05.295100 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:05.295205 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:05.295434 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:00:05.295440 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:00:05.295592 (+ 152us) client_negotiation.cc:769] Sending connection context
0926 15:00:05.295710 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 15:00:05.295873 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":2,"thread_start_us":63,"threads_started":1}
I20260926 15:00:05.296759 30066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.288475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:40427 (local address 127.0.0.1:33152)
0926 15:00:05.288521 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.288532 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 15:00:05.288564 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:00:05.288955 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:00:05.288956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:00:05.289113 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:00:05.289153 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:05.289155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.290299 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.290300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:05.290783 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:05.290784 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.291233 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.291234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:05.291290 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.291772 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:00:05.291776 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:00:05.293417 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:00:05.295548 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:05.295549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:05.295549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:05.295779 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:05.295979 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:05.295979 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:05.295980 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:05.296033 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:05.296484 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:00:05.296485 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:00:05.296530 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 15:00:05.296599 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 15:00:05.296689 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 15:00:05.296785 30143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.288531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33152 (local address 127.7.37.252:40427)
0926 15:00:05.288663 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.288672 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.288696 (+ 24us) server_negotiation.cc:407] Connection header received
0926 15:00:05.288823 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.288844 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.288863 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.288939 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.289252 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.290216 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.290855 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.291160 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.291232 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.293798 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:05.293806 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:05.293806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:05.293812 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:05.295473 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.295856 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.295859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.295859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.295910 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.296275 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.296276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.296276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.296426 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:05.296499 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:05.296626 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 15:00:05.296698 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260926 15:00:05.297096 30140 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:00:05 dist-test-slave-f9xb krb5kdc[29908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.252@KRBTEST.COM
I20260926 15:00:05.298614 30064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3: Bootstrap starting.
I20260926 15:00:05.299576 30064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3: Neither blocks nor log segments found. Creating new log.
I20260926 15:00:05.299896 30064 log.cc:824] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3: Log is configured to *not* fsync() on all Append() calls
I20260926 15:00:05.300674 30064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3: No bootstrap required, opened a new log
Sep 26 15:00:05 dist-test-slave-f9xb krb5kdc[29908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434805, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.252@KRBTEST.COM for kudu/127.7.37.253@KRBTEST.COM
I20260926 15:00:05.301478 30144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.292991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33160 (local address 127.7.37.252:40427)
0926 15:00:05.293361 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.293363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.293480 (+ 117us) server_negotiation.cc:407] Connection header received
0926 15:00:05.293571 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.293572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.293582 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.293654 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.294218 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.294894 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.295875 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.296150 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.296206 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.298764 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:05.298765 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:05.298766 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:05.298772 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:05.300320 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.300656 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.300657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.300657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.300718 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.300906 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.300907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.300907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.301004 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:05.301119 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:05.301300 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 15:00:05.301365 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":68,"threads_started":1}
I20260926 15:00:05.302675 30064 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb76135da89943639740f1afd56583a4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } } peers { permanent_uuid: "cfd605b154884ef9aba07963570a39d3" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } } peers { permanent_uuid: "cbc3d0f76861414d889d7ae54ee8d033" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } }
I20260926 15:00:05.302812 30064 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd605b154884ef9aba07963570a39d3, State: Initialized, Role: FOLLOWER
I20260926 15:00:05.302887 30064 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb76135da89943639740f1afd56583a4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } } peers { permanent_uuid: "cfd605b154884ef9aba07963570a39d3" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } } peers { permanent_uuid: "cbc3d0f76861414d889d7ae54ee8d033" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } }
I20260926 15:00:05.303193 30064 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:00:05.303154 30147 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb76135da89943639740f1afd56583a4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } } peers { permanent_uuid: "cfd605b154884ef9aba07963570a39d3" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } } peers { permanent_uuid: "cbc3d0f76861414d889d7ae54ee8d033" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } } }
I20260926 15:00:05.303243 30147 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:00:05.304008 30142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.297310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.253:43841 (local address 127.0.0.1:46038)
0926 15:00:05.297338 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.297347 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 15:00:05.297386 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:00:05.297501 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:00:05.297502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:00:05.297625 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:00:05.297672 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:05.297673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.298282 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.298283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:05.298829 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:05.298831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.299160 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.299161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:05.299228 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.299959 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:00:05.299964 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:00:05.301280 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:00:05.302934 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:05.302935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:05.302936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:05.303119 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:05.303338 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:05.303339 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:05.303339 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:05.303397 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:05.303712 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:00:05.303713 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:00:05.303781 (+ 68us) client_negotiation.cc:769] Sending connection context
0926 15:00:05.303841 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 15:00:05.303918 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260926 15:00:05.304270 30068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.297371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46038 (local address 127.7.37.253:43841)
0926 15:00:05.297428 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.297430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.297435 (+ 5us) server_negotiation.cc:407] Connection header received
0926 15:00:05.297449 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.297449 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.297457 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.297500 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.297758 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.298207 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.298902 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.299101 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.299152 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.301371 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:05.301372 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:05.301372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:05.301378 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:05.302850 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.303194 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.303194 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.303195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.303270 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.303466 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.303467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.303468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.303624 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:05.303683 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:05.304119 (+ 436us) server_negotiation.cc:299] Negotiation successful
0926 15:00:05.304196 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 15:00:05.304515 30140 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } has no permanent_uuid. Determining permanent_uuid...
W20260926 15:00:05.305194 30158 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 15:00:05.305244 30158 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 26 15:00:05 dist-test-slave-f9xb krb5kdc[29908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.254@KRBTEST.COM
I20260926 15:00:05.305697 29996 master_runner.cc:186] Error getting master registration for 127.7.37.254:32829: OK, instance_id { permanent_uuid: "fb76135da89943639740f1afd56583a4" instance_seqno: 1790434804997186 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 15:00:05.308368 29996 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 26 15:00:05 dist-test-slave-f9xb krb5kdc[29908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434805, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.252@KRBTEST.COM for kudu/127.7.37.252@KRBTEST.COM
I20260926 15:00:05.309523 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.302083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34330 (local address 127.7.37.254:32829)
0926 15:00:05.302115 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.302117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.302125 (+ 8us) server_negotiation.cc:407] Connection header received
0926 15:00:05.302367 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.302368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.302378 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.302417 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.302856 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.303409 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.304477 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.304681 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.304739 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.306969 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:05.306970 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:05.306970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:05.306976 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:05.308503 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.308839 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.308840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.308841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.308891 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.309070 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.309071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.309071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.309172 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:05.309306 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:05.309376 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 15:00:05.309441 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 15:00:05.312637 30142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.304711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:40427 (local address 127.0.0.1:33166)
0926 15:00:05.304746 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.304754 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 15:00:05.304785 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:00:05.306024 (+ 1239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:00:05.306025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:00:05.306153 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:00:05.306184 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:05.306186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.307001 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.307002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:05.307676 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:05.307678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.308134 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.308135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:05.308178 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.308502 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:00:05.308506 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:00:05.310030 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:00:05.311662 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:05.311663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:05.311663 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:05.311896 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:05.312079 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:05.312080 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:05.312080 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:05.312140 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:05.312402 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:00:05.312402 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:00:05.312435 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 15:00:05.312475 (+ 40us) client_negotiation.cc:240] Negotiation successful
0926 15:00:05.312552 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260926 15:00:05.312965 30144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.304858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33166 (local address 127.7.37.252:40427)
0926 15:00:05.305012 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.305013 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.305045 (+ 32us) server_negotiation.cc:407] Connection header received
0926 15:00:05.305085 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.305086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.305097 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.306245 (+ 1148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.306265 (+ 20us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.306932 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.307757 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.308056 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.308296 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.310115 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:05.310117 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:05.310117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:05.310123 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:05.311582 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.311964 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.311965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.311965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.312021 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.312200 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.312200 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.312201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.312329 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:05.312789 (+ 460us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:05.312825 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 15:00:05.312891 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260926 15:00:05.314078 30140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033: Bootstrap starting.
Sep 26 15:00:05 dist-test-slave-f9xb krb5kdc[29908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.37.253@KRBTEST.COM
I20260926 15:00:05.314729 30140 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033: Neither blocks nor log segments found. Creating new log.
I20260926 15:00:05.314966 30140 log.cc:824] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033: Log is configured to *not* fsync() on all Append() calls
I20260926 15:00:05.315781 30140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033: No bootstrap required, opened a new log
I20260926 15:00:05.317312 30068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.310056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46052 (local address 127.7.37.253:43841)
0926 15:00:05.310087 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.310088 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.310149 (+ 61us) server_negotiation.cc:407] Connection header received
0926 15:00:05.310214 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.310214 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.310223 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.310265 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.310752 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.311270 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.312001 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.312279 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.312329 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.314616 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:05.314617 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:05.314617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:05.314622 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:05.316079 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.316476 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.316477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.316478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.316534 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.316735 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.316735 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.316736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.316856 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:05.316981 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:05.317068 (+ 87us) server_negotiation.cc:299] Negotiation successful
0926 15:00:05.317136 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260926 15:00:05.317616 30140 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb76135da89943639740f1afd56583a4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } } peers { permanent_uuid: "cfd605b154884ef9aba07963570a39d3" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } } peers { permanent_uuid: "cbc3d0f76861414d889d7ae54ee8d033" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } }
I20260926 15:00:05.317767 30140 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbc3d0f76861414d889d7ae54ee8d033, State: Initialized, Role: FOLLOWER
I20260926 15:00:05.317848 30140 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb76135da89943639740f1afd56583a4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } } peers { permanent_uuid: "cfd605b154884ef9aba07963570a39d3" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } } peers { permanent_uuid: "cbc3d0f76861414d889d7ae54ee8d033" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } }
I20260926 15:00:05.318080 30161 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb76135da89943639740f1afd56583a4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } } peers { permanent_uuid: "cfd605b154884ef9aba07963570a39d3" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } } peers { permanent_uuid: "cbc3d0f76861414d889d7ae54ee8d033" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } } }
I20260926 15:00:05.318164 30161 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:00:05.318264 30140 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:00:05.318538 7319 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:00:05.319592 29963 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "fb76135da89943639740f1afd56583a4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34330
W20260926 15:00:05.320142 30172 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 15:00:05.320820 30172 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 15:00:05.321158 30071 master_runner.cc:186] Error getting master registration for 127.7.37.254:32829: OK, instance_id { permanent_uuid: "fb76135da89943639740f1afd56583a4" instance_seqno: 1790434804997186 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 15:00:05.323251 30071 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:00:05.336676 29989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } attempt: 2
Sep 26 15:00:05 dist-test-slave-f9xb krb5kdc[29908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434804, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.37.254@KRBTEST.COM for kudu/127.7.37.252@KRBTEST.COM
I20260926 15:00:05.344117 30144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.337048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33176 (local address 127.7.37.252:40427)
0926 15:00:05.337085 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.337086 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.337094 (+ 8us) server_negotiation.cc:407] Connection header received
0926 15:00:05.337114 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.337114 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.337124 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.337190 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.337482 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.338201 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.339160 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.339467 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.339536 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.341497 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:05.341498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:05.341499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:05.341506 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:05.343035 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.343395 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.343396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.343397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.343451 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.343625 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.343625 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.343626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.343774 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:05.343836 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:05.343956 (+ 120us) server_negotiation.cc:299] Negotiation successful
0926 15:00:05.344029 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 15:00:05.344120 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.336929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.37.252:40427 (local address 127.0.0.1:33176)
0926 15:00:05.336974 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 15:00:05.336984 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 15:00:05.337018 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:00:05.337205 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:00:05.337206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:00:05.337363 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:00:05.337409 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:05.337411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.338300 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.338302 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:05.339085 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:00:05.339086 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.339545 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.339546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:00:05.339602 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.340041 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:00:05.340047 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:00:05.341424 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:00:05.343121 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:05.343121 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:05.343122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:05.343333 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:05.343516 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:00:05.343517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:00:05.343517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:00:05.343577 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:00:05.343826 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:00:05.343827 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:00:05.343870 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 15:00:05.343934 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 15:00:05.344021 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 15:00:05.345525 29989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4: Bootstrap starting.
I20260926 15:00:05.346187 29989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4: Neither blocks nor log segments found. Creating new log.
I20260926 15:00:05.346468 29989 log.cc:824] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4: Log is configured to *not* fsync() on all Append() calls
I20260926 15:00:05.347357 29989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4: No bootstrap required, opened a new log
I20260926 15:00:05.348656 29989 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb76135da89943639740f1afd56583a4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } } peers { permanent_uuid: "cfd605b154884ef9aba07963570a39d3" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } } peers { permanent_uuid: "cbc3d0f76861414d889d7ae54ee8d033" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } }
I20260926 15:00:05.348788 29989 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb76135da89943639740f1afd56583a4, State: Initialized, Role: FOLLOWER
I20260926 15:00:05.348865 29989 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb76135da89943639740f1afd56583a4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } } peers { permanent_uuid: "cfd605b154884ef9aba07963570a39d3" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } } peers { permanent_uuid: "cbc3d0f76861414d889d7ae54ee8d033" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } }
I20260926 15:00:05.349145 30175 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb76135da89943639740f1afd56583a4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } } peers { permanent_uuid: "cfd605b154884ef9aba07963570a39d3" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } } peers { permanent_uuid: "cbc3d0f76861414d889d7ae54ee8d033" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } } }
I20260926 15:00:05.349224 29963 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 15:00:05.349150 29989 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:00:05.349401 29963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:00:05.350198 29963 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb76135da89943639740f1afd56583a4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } } peers { permanent_uuid: "cfd605b154884ef9aba07963570a39d3" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } } peers { permanent_uuid: "cbc3d0f76861414d889d7ae54ee8d033" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } }
I20260926 15:00:05.350414 29963 leader_election.cc:288] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [CANDIDATE]: Term 1 election: Requested vote from peers cfd605b154884ef9aba07963570a39d3 (127.7.37.253:43841), cbc3d0f76861414d889d7ae54ee8d033 (127.7.37.252:40427)
I20260926 15:00:05.351786 30038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fb76135da89943639740f1afd56583a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cfd605b154884ef9aba07963570a39d3"
I20260926 15:00:05.351912 30038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:00:05.352728 30038 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb76135da89943639740f1afd56583a4 in term 1.
I20260926 15:00:05.353142 29939 leader_election.cc:302] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cfd605b154884ef9aba07963570a39d3, fb76135da89943639740f1afd56583a4; no voters:
I20260926 15:00:05.353616 30114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fb76135da89943639740f1afd56583a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cbc3d0f76861414d889d7ae54ee8d033"
I20260926 15:00:05.353730 30114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:00:05.354513 30114 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb76135da89943639740f1afd56583a4 in term 1.
I20260926 15:00:05.356729 29921 master_runner.cc:263] No leader master found from master fb76135da89943639740f1afd56583a4
I20260926 15:00:05.356771 29921 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 15:00:05.351399 30186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 15:00:05.356886 30186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 15:00:05.349363 30175 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:00:05.357024 30175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:00:05.357100 30175 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [term 1 LEADER]: Becoming Leader. State: Replica: fb76135da89943639740f1afd56583a4, State: Running, Role: LEADER
I20260926 15:00:05.357215 30175 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb76135da89943639740f1afd56583a4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } } peers { permanent_uuid: "cfd605b154884ef9aba07963570a39d3" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } } peers { permanent_uuid: "cbc3d0f76861414d889d7ae54ee8d033" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } }
I20260926 15:00:05.357681 30175 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb76135da89943639740f1afd56583a4. Latest consensus state: current_term: 1 leader_uuid: "fb76135da89943639740f1afd56583a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb76135da89943639740f1afd56583a4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } } peers { permanent_uuid: "cfd605b154884ef9aba07963570a39d3" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } } peers { permanent_uuid: "cbc3d0f76861414d889d7ae54ee8d033" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } } }
I20260926 15:00:05.357793 30175 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [sys.catalog]: This master's current role is: LEADER
I20260926 15:00:05.358901 30194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:00:05.359491 30194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:00:05.372138 30114 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033 [term 1 FOLLOWER]: Refusing update from remote peer fb76135da89943639740f1afd56583a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:00:05.372605 30175 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbc3d0f76861414d889d7ae54ee8d033" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:00:05.373906 30161 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb76135da89943639740f1afd56583a4. Latest consensus state: current_term: 1 leader_uuid: "fb76135da89943639740f1afd56583a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb76135da89943639740f1afd56583a4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } } peers { permanent_uuid: "cfd605b154884ef9aba07963570a39d3" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } } peers { permanent_uuid: "cbc3d0f76861414d889d7ae54ee8d033" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } } }
I20260926 15:00:05.373988 30161 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:00:05.374395 30038 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3 [term 1 FOLLOWER]: Refusing update from remote peer fb76135da89943639740f1afd56583a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:00:05.374661 30192 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfd605b154884ef9aba07963570a39d3" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:00:05.375778 30147 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb76135da89943639740f1afd56583a4. Latest consensus state: current_term: 1 leader_uuid: "fb76135da89943639740f1afd56583a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb76135da89943639740f1afd56583a4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } } peers { permanent_uuid: "cfd605b154884ef9aba07963570a39d3" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } } peers { permanent_uuid: "cbc3d0f76861414d889d7ae54ee8d033" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } } }
I20260926 15:00:05.375859 30147 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:00:05.385152 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.358498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34342 (local address 127.7.37.254:32829)
0926 15:00:05.358531 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.358532 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.360767 (+ 2235us) server_negotiation.cc:407] Connection header received
0926 15:00:05.360847 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.360848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.360857 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.360883 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.363809 (+ 2926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.364517 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.368534 (+ 4017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.368821 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.368860 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.379658 (+ 10798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:05.379660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:05.379660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:05.379666 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:05.381216 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.381832 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.381833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.381834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.381883 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.384220 (+ 2337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.384221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.384221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.384351 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:05.384427 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:05.384984 (+ 557us) server_negotiation.cc:299] Negotiation successful
0926 15:00:05.385060 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 15:00:05.386116 30068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.358116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46060 (local address 127.7.37.253:43841)
0926 15:00:05.358151 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.358152 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.360304 (+ 2152us) server_negotiation.cc:407] Connection header received
0926 15:00:05.360395 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.360396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.360405 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.360432 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.362722 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.363458 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.369368 (+ 5910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.369667 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.369706 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.378040 (+ 8334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:05.378041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:05.378041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:05.378048 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:05.379559 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.382063 (+ 2504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.382064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.382064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.382111 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.384593 (+ 2482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.384594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.384594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.384723 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:05.384801 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:05.385950 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0926 15:00:05.386026 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 15:00:05.387656 30144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:05.358764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33180 (local address 127.7.37.252:40427)
0926 15:00:05.358805 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.358806 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.361952 (+ 3146us) server_negotiation.cc:407] Connection header received
0926 15:00:05.362034 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.362035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.362043 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.362069 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.365827 (+ 3758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.366521 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.376068 (+ 9547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:05.376373 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:05.376413 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:05.382305 (+ 5892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:00:05.382307 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:00:05.382307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:00:05.382313 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:00:05.383866 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.386868 (+ 3002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.386869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.386870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.386927 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:00:05.387129 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:00:05.387130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:00:05.387130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:00:05.387258 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:00:05.387328 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:05.387501 (+ 173us) server_negotiation.cc:299] Negotiation successful
0926 15:00:05.387570 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 15:00:05.388746 30175 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fb76135da89943639740f1afd56583a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb76135da89943639740f1afd56583a4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } } peers { permanent_uuid: "cfd605b154884ef9aba07963570a39d3" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } } peers { permanent_uuid: "cbc3d0f76861414d889d7ae54ee8d033" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } } }
I20260926 15:00:05.388842 30175 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [sys.catalog]: This master's current role is: LEADER
I20260926 15:00:05.389184 30147 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fb76135da89943639740f1afd56583a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb76135da89943639740f1afd56583a4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } } peers { permanent_uuid: "cfd605b154884ef9aba07963570a39d3" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } } peers { permanent_uuid: "cbc3d0f76861414d889d7ae54ee8d033" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } } }
I20260926 15:00:05.389272 30147 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:00:05.389487 30198 mvcc.cc:205] Tried to move back new op lower bound from 7333620962788409344 to 7333620962744127488. Current Snapshot: MvccSnapshot[applied={T|T < 7333620962788409344}]
I20260926 15:00:05.389492 30201 mvcc.cc:205] Tried to move back new op lower bound from 7333620962788409344 to 7333620962744127488. Current Snapshot: MvccSnapshot[applied={T|T < 7333620962788409344}]
I20260926 15:00:05.390092 30192 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fb76135da89943639740f1afd56583a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb76135da89943639740f1afd56583a4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } } peers { permanent_uuid: "cfd605b154884ef9aba07963570a39d3" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } } peers { permanent_uuid: "cbc3d0f76861414d889d7ae54ee8d033" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } } }
I20260926 15:00:05.390198 30192 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4 [sys.catalog]: This master's current role is: LEADER
I20260926 15:00:05.390619 30161 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fb76135da89943639740f1afd56583a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb76135da89943639740f1afd56583a4" member_type: VOTER last_known_addr { host: "127.7.37.254" port: 32829 } } peers { permanent_uuid: "cfd605b154884ef9aba07963570a39d3" member_type: VOTER last_known_addr { host: "127.7.37.253" port: 43841 } } peers { permanent_uuid: "cbc3d0f76861414d889d7ae54ee8d033" member_type: VOTER last_known_addr { host: "127.7.37.252" port: 40427 } } }
I20260926 15:00:05.390693 30161 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:00:05.391016 30199 mvcc.cc:205] Tried to move back new op lower bound from 7333620962788409344 to 7333620962744127488. Current Snapshot: MvccSnapshot[applied={T|T < 7333620962788409344}]
I20260926 15:00:05.396204 30194 catalog_manager.cc:1347] Generated new cluster ID: a5ab85818d43454cab2de8b1f48e61c8
I20260926 15:00:05.396245 30194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:00:05.413190 30194 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:00:05.413933 30194 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:00:05.420609 30194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb76135da89943639740f1afd56583a4: Generated new TSK 0
I20260926 15:00:05.420840 30194 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 15:00:05.440568 30068 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:05.438964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46062 (local address 127.7.37.253:43841)
0926 15:00:05.439006 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.439008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.439810 (+ 802us) server_negotiation.cc:407] Connection header received
0926 15:00:05.440044 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.440046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.440057 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.440086 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.440501 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46062: BlockingRecv error: recv got EOF from 127.0.0.1:46062 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260926 15:00:05.441424 29992 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:05.439354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34358 (local address 127.7.37.254:32829)
0926 15:00:05.439389 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.439391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.440945 (+ 1554us) server_negotiation.cc:407] Connection header received
0926 15:00:05.441032 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.441033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.441044 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.441070 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.441366 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34358: BlockingRecv error: recv got EOF from 127.0.0.1:34358 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260926 15:00:05.442216 30144 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:05.439641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33196 (local address 127.7.37.252:40427)
0926 15:00:05.439677 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.439679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.441716 (+ 2037us) server_negotiation.cc:407] Connection header received
0926 15:00:05.441829 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.441830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.441839 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.441864 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.442126 (+ 262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33196: BlockingRecv error: recv got EOF from 127.0.0.1:33196 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260926 15:00:05.444446 30068 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:05.443347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46074 (local address 127.7.37.253:43841)
0926 15:00:05.443381 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.443382 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.443963 (+ 581us) server_negotiation.cc:407] Connection header received
0926 15:00:05.444072 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.444073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.444088 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.444115 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.444400 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46074: BlockingRecv error: recv got EOF from 127.0.0.1:46074 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260926 15:00:05.447294 30068 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:05.446596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46084 (local address 127.7.37.253:43841)
0926 15:00:05.446626 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.446628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.446842 (+ 214us) server_negotiation.cc:407] Connection header received
0926 15:00:05.446928 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.446929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.446937 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.446962 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.447250 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46084: BlockingRecv error: recv got EOF from 127.0.0.1:46084 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260926 15:00:05.451102 30068 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:05.450381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46090 (local address 127.7.37.253:43841)
0926 15:00:05.450416 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.450417 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.450611 (+ 194us) server_negotiation.cc:407] Connection header received
0926 15:00:05.450716 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.450717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.450727 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.450755 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.451060 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46090: BlockingRecv error: recv got EOF from 127.0.0.1:46090 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260926 15:00:05.457007 30068 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:05.456278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46102 (local address 127.7.37.253:43841)
0926 15:00:05.456309 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.456311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.456540 (+ 229us) server_negotiation.cc:407] Connection header received
0926 15:00:05.456629 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.456630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.456640 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.456667 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.456964 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46102: BlockingRecv error: recv got EOF from 127.0.0.1:46102 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 15:00:05.467072 30068 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:05.466298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46108 (local address 127.7.37.253:43841)
0926 15:00:05.466346 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.466348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.466592 (+ 244us) server_negotiation.cc:407] Connection header received
0926 15:00:05.466677 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.466678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.466688 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.466714 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.467018 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46108: BlockingRecv error: recv got EOF from 127.0.0.1:46108 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260926 15:00:05.484941 30068 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:05.484401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46118 (local address 127.7.37.253:43841)
0926 15:00:05.484444 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.484446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.484455 (+ 9us) server_negotiation.cc:407] Connection header received
0926 15:00:05.484519 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.484519 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.484529 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.484576 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.484888 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46118: BlockingRecv error: recv got EOF from 127.0.0.1:46118 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260926 15:00:05.518646 30068 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:05.517972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46132 (local address 127.7.37.253:43841)
0926 15:00:05.518026 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.518028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.518126 (+ 98us) server_negotiation.cc:407] Connection header received
0926 15:00:05.518220 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.518222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.518232 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.518279 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.518584 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46132: BlockingRecv error: recv got EOF from 127.0.0.1:46132 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260926 15:00:05.584283 30068 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:05.583612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46138 (local address 127.7.37.253:43841)
0926 15:00:05.583675 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.583684 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.583766 (+ 82us) server_negotiation.cc:407] Connection header received
0926 15:00:05.583848 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.583849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.583861 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.583909 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.584236 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46138: BlockingRecv error: recv got EOF from 127.0.0.1:46138 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260926 15:00:05.714284 30068 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:05.713522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46140 (local address 127.7.37.253:43841)
0926 15:00:05.713589 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.713591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.713713 (+ 122us) server_negotiation.cc:407] Connection header received
0926 15:00:05.713792 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.713793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.713804 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.713851 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.714221 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46140: BlockingRecv error: recv got EOF from 127.0.0.1:46140 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260926 15:00:05.972286 30068 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:00:05.971624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46156 (local address 127.7.37.253:43841)
0926 15:00:05.971677 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:05.971679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:05.971705 (+ 26us) server_negotiation.cc:407] Connection header received
0926 15:00:05.971789 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:05.971790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:05.971800 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:05.971855 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:00:05.972231 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46156: BlockingRecv error: recv got EOF from 127.0.0.1:46156 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 15:00:06.305630 30158 catalog_manager.cc:1259] Loaded cluster ID: a5ab85818d43454cab2de8b1f48e61c8
I20260926 15:00:06.305696 30158 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3: loading cluster ID for follower catalog manager: success
I20260926 15:00:06.306443 30158 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3: acquiring CA information for follower catalog manager: success
I20260926 15:00:06.306686 30158 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cfd605b154884ef9aba07963570a39d3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:00:06.321172 30172 catalog_manager.cc:1259] Loaded cluster ID: a5ab85818d43454cab2de8b1f48e61c8
I20260926 15:00:06.321245 30172 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033: loading cluster ID for follower catalog manager: success
I20260926 15:00:06.321988 30172 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033: acquiring CA information for follower catalog manager: success
I20260926 15:00:06.322244 30172 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cbc3d0f76861414d889d7ae54ee8d033: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:00:06.487560 30270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:06.485307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46172 (local address 127.7.37.253:43841)
0926 15:00:06.485392 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:06.485394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:06.485403 (+ 9us) server_negotiation.cc:407] Connection header received
0926 15:00:06.485470 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:06.485471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:06.485515 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:06.485575 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:06.485702 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:06.486184 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:06.486882 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:06.487066 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:06.487113 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:06.487221 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:06.487322 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:06.487375 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:06.487463 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 15:00:06.487479 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70,"thread_start_us":46,"threads_started":1}
I20260926 15:00:06.491499 30276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:06.489269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33206 (local address 127.7.37.252:40427)
0926 15:00:06.489381 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:06.489383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:06.489389 (+ 6us) server_negotiation.cc:407] Connection header received
0926 15:00:06.489426 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:06.489427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:06.489461 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:06.489522 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:06.489632 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:06.490118 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:06.490803 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:06.491008 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:06.491059 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:06.491201 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:06.491298 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:06.491357 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:06.491399 (+ 42us) server_negotiation.cc:299] Negotiation successful
0926 15:00:06.491419 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":44,"threads_started":1}
I20260926 15:00:06.495451 30270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:06.492976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46186 (local address 127.7.37.253:43841)
0926 15:00:06.493009 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:06.493011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:06.493069 (+ 58us) server_negotiation.cc:407] Connection header received
0926 15:00:06.493092 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:06.493101 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:06.493110 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:06.493154 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:06.493264 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:06.493972 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:06.494732 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:06.494997 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:06.495055 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:06.495213 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:06.495243 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:06.495287 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:06.495366 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 15:00:06.495389 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 15:00:06.495807 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:06.492858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34362 (local address 127.7.37.254:32829)
0926 15:00:06.492959 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:06.492961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:06.492969 (+ 8us) server_negotiation.cc:407] Connection header received
0926 15:00:06.493008 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:06.493009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:06.493057 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:06.493310 (+ 253us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:06.493350 (+ 40us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:06.493835 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:06.494778 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:06.495074 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:06.495128 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:06.495460 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:06.495555 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:06.495635 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:06.495690 (+ 55us) server_negotiation.cc:299] Negotiation successful
0926 15:00:06.495715 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":52,"threads_started":1}
I20260926 15:00:06.496076 30276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:00:06.493099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33216 (local address 127.7.37.252:40427)
0926 15:00:06.493198 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0926 15:00:06.493199 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 15:00:06.493206 (+ 7us) server_negotiation.cc:407] Connection header received
0926 15:00:06.493250 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:00:06.493251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:00:06.493259 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:00:06.493299 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:00:06.493406 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:06.494067 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:06.494974 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:00:06.495217 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:00:06.495274 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:00:06.495480 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:00:06.495505 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:00:06.495832 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0926 15:00:06.495890 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 15:00:06.495912 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 15:00:06.497304 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 29921
I20260926 15:00:06.501189 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 29996
I20260926 15:00:06.505239 7319 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrMzbz4/build/release/bin/kudu with pid 30071
2026-09-26T15:00:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2669 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5366 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (251497 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST